Video encoding method, video decoding method, and bitstream transmission method

By dividing the maximum codec unit and dividing it according to the maximum transform size, the problem of high-resolution image encoding and decoding is solved, which improves the encoding and decoding efficiency and reduces the complexity.

CN119094759APending Publication Date: 2024-12-06SAMSUNG ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411277005.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2018-12-21
Filing Date
2019-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has high encoding and decoding complexity when encoding and decoding high-resolution or high-definition images, making it difficult to improve efficiency and reduce complexity.

Method used

By dividing at least one of the height and width of the maximum codec unit, the codec unit is generated, and inversely transform or transform according to the maximum transform size, it is determined whether binary or quad-tree division is allowed to be performed to optimize the codec process.

Benefits of technology

Improves the efficiency of image encoding and decoding, reduces complexity, and enhances the performance of pipeline data units designed by hardware.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119094759A_ABST
    Figure CN119094759A_ABST
Patent Text Reader

Abstract

Provided is a video decoding method, comprising: generating a codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size; determining whether two second codec units are allowed to be generated by dividing at least one of a height and a width of a first codec unit based on whether a height or a width of the non-square first codec unit including an outer boundary of the image among the codec units is greater than a maximum transform size; and decoding a second codec unit generated from the first codec unit.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional application, the application number of the parent application is: 201980069515.8, the application date is: August 28, 2019, and the name of the invention is: Video encoding method and encoding device considering hardware design, as well as video decoding method and decoding device. Technical Field

[0002] The method and apparatus according to the embodiment may encode or decode an image by using coding units having various shapes included in the image. Background Art

[0003] With the development and popularization of hardware capable of reproducing and storing high-resolution or high-definition image content, the demand for codecs that effectively encode or decode high-resolution or high-definition image content has increased. The encoded image content can be decoded and then reproduced. Recently, methods for effectively compressing such high-resolution or high-definition image content are being used. For example, methods for effectively achieving image compression by dividing the processing of an image to be encoded by an arbitrary method have been implemented.

[0004] Various data units may be used to compress an image, and there may be an inclusion relationship between these data units. In order to determine the size of the data unit to be used for image compression, the data unit may be divided by using various methods, and the image may be encoded or decoded by determining the data unit optimized based on the characteristics of the image.

[0005] The data unit can be determined by recursively performing flexible tree partitioning. Flexible tree partitioning may include binary split, ternary split, or quadtree splitting. In addition, by allowing square data units and non-square data units, data units optimized for encoding can be determined according to the characteristics of the image. However, because various partition shapes and various data unit shapes are used, the encoding complexity may increase. Therefore, it is necessary to improve the encoding and decoding efficiency and reduce the complexity of the image decoding / encoding method and device by using flexible tree partitioning. Summary of the invention

[0006] Technical issues

[0007] Provided are an image decoding / encoding method and device, which consider the size of a pipeline data unit used for hardware design, and improve encoding / encoding efficiency and reduce complexity by determining a block division method.

[0008] Technical Solution

[0009] In order to overcome the above technical problems, the video decoding method proposed in the present invention includes: generating a codec unit by dividing at least one of the height and the width of a maximum codec unit having a first size; determining whether it is allowed to generate two second codec units by dividing at least one of the height and the width of the first codec unit based on whether the height or the width of the non-square first codec unit including the outer boundary of the image in the codec unit is greater than the maximum transformation size; and decoding the second codec unit generated from the first codec unit by inverse transforming the second codec unit using the maximum transformation size.

[0010] Determining whether to allow the generation of two second codec units by dividing at least one of the height and the width of the first codec unit may include: when the length of the height of the first codec unit is greater than the maximum transform size and the length of the width of the first codec unit is equal to the maximum transform size, allowing the generation of a smaller codec unit by performing horizontal binary splitting on the first codec unit; and prohibiting the generation of a smaller codec unit by performing vertical binary splitting on the first codec unit.

[0011] Determining whether to allow the generation of two second codec units by dividing at least one of the height and the width of the first codec unit may include: when the length of the height of the first codec unit is equal to the maximum transformation size and the length of the width of the first codec unit is greater than the maximum transformation size, allowing the generation of a smaller codec unit by performing vertical binary division on the first codec unit; and prohibiting the generation of a smaller codec unit by performing horizontal binary division on the first codec unit.

[0012] The video decoding method may further include, when the length of the height of the codec unit including the outer boundary of the image is greater than the maximum transform size and the length of the width of the codec unit is greater than the maximum transform size, allowing a smaller codec unit to be generated by performing quadtree partitioning on the codec unit.

[0013] The length of the height and width of the codec unit can be greater than or equal to the maximum transform size.

[0014] Generating the codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size may include determining the maximum codec unit of the first size by dividing an image based on information about the size of the maximum codec unit obtained from a bitstream.

[0015] Generating a codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size may include: obtaining from a bitstream information about a minimum size of a luma codec unit, information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing binary division on an intra slice, information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing binary division on an inter slice, information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing trifurcating division on an intra slice, and information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing trifurcating division on an inter slice; determining a maximum size of a luma codec unit generated by performing binary division by using the information about the minimum size of the luma codec unit and the information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing binary division on an intra slice, the binary division being determined in an intra slice; determining a maximum size of a luma codec unit generated by performing binary division by using the information about the minimum size of the luma codec unit and the information about a difference between a maximum size and a minimum size of a luma codec unit generated by performing trifurcating division on an inter slice The method further comprises: determining a maximum size of a luma codec unit generated by performing a three-way division using information about a difference between a maximum size and a minimum size of the codec unit, the three-way division being determined in an intra-frame slice; determining a maximum size of a luma codec unit generated by performing a binary division using information about a minimum size of the luma codec unit and information about a difference between a maximum size and a minimum size of the luma codec unit generated by performing a binary division on an inter-frame slice, the binary division being determined in an inter-frame slice; and determining a maximum size of a luma codec unit generated by performing a three-way division using information about a minimum size of the luma codec unit and information about a difference between a maximum size and a minimum size of the luma codec unit generated by performing a three-way division on an inter-frame slice, the three-way division being determined in an inter-frame slice.

[0016] Generating a codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size may include: obtaining information indicating whether a luma codec unit and a chroma codec unit are determined separately from a bitstream; when the luma codec unit and the chroma codec unit are determined separately according to the obtained information, obtaining information about a difference between a maximum size and a minimum size of a chroma codec unit generated by performing binary division on an intra slice and information about a difference between a maximum size and a minimum size of a chroma codec unit generated by performing trifurcating division on the intra slice from the bitstream; and obtaining information about a difference between a maximum size and a minimum size of a chroma codec unit generated by performing trifurcating division on the intra slice by using information about the chroma codec unit. The maximum size of the chroma codec unit generated by performing binary partitioning is determined by using information about a minimum size of the chroma codec unit and information about a difference between a maximum size and a minimum size of the chroma codec unit generated by performing binary partitioning on the intra slice, the binary partitioning being determined in the intra slice; and the maximum size of the chroma codec unit generated by performing three-way partitioning is determined by using information about the minimum size of the chroma codec unit and information about a difference between a maximum size and a minimum size of the chroma codec unit generated by performing three-way partitioning on the intra slice, the three-way partitioning being determined in the intra slice.

[0017] The video decoding method may further include: when a first codec unit in the codec units is not adjacent to an outer boundary of the image, obtaining division type information of the first codec unit; when the division type information of the first codec unit indicates a binary division, dividing the height or width of the first codec unit in half and determining two second codec units, each of which has a depth that is increased by 1 compared to the depth of the first codec unit; and when the division type information of the first codec unit indicates a ternary division, dividing the height or width of the first codec unit at a ratio of 1:2:1, and determining a third codec unit and two fourth codec units, the depth of the third codec unit being increased by 1 compared to the depth of the first codec unit, and the depth of each fourth codec unit being increased by 2 compared to the depth of the first codec unit.

[0018] In order to overcome the above technical problems, the video decoding device proposed by the present invention includes: a processor, which is configured to determine a codec unit by dividing at least one of the height and the width of a maximum codec unit having a first size; based on whether the height or the width of a non-square first codec unit including an outer boundary of an image in the codec unit is greater than the maximum transformation size, determine whether it is allowed to generate two second codec units by dividing at least one of the height and the width of the first codec unit; and decode the second codec unit generated from the first codec unit by inverse transforming the second codec unit using the maximum transformation size; and a memory for storing data generated by the processor.

[0019] In order to overcome the above technical problems, the video encoding method proposed in the present invention includes: determining a codec unit by dividing at least one of the height and the width of a maximum codec unit having a first size; determining whether it is allowed to generate two second codec units by dividing at least one of the height and the width of the first codec unit based on whether the height or the width of a non-square first codec unit including an outer boundary of an image in the codec unit is greater than a maximum transformation size; and encoding the second codec unit generated from the first codec unit by transforming the second codec unit using the maximum transformation size.

[0020] Determining whether to allow the generation of two second codec units by dividing at least one of the height and the width of the first codec unit may include: when the length of the height of the first codec unit is greater than the maximum transform size and the length of the width of the first codec unit is equal to the maximum transform size, allowing the generation of a smaller codec unit by performing horizontal binary splitting on the first codec unit; and prohibiting the generation of a smaller codec unit by performing vertical binary splitting on the first codec unit.

[0021] Determining whether to allow the generation of two second codec units by dividing at least one of the height and the width of the first codec unit may include: when the length of the height of the first codec unit is equal to the maximum transformation size and the length of the width of the first codec unit is greater than the maximum transformation size, allowing the generation of a smaller codec unit by performing vertical binary division on the first codec unit; and prohibiting the generation of a smaller codec unit by performing horizontal binary division on the first codec unit.

[0022] The video encoding method may further include: when the length of the height of the codec unit including the outer boundary of the image is greater than the maximum transform size and the length of the width of the codec unit is greater than the maximum transform size, allowing a smaller codec unit to be generated by performing quadtree partitioning on the codec unit.

[0023] Generating the codec unit by dividing at least one of a height and a width of the maximum codec unit having the first size may include determining the maximum codec unit having the first size by dividing the image, and encoding information about the size of the maximum codec unit.

[0024] The video encoding method may further include: determining information about a difference between the maximum size and the minimum size of a luma codec unit generated by performing the binary division on the intra-frame slice by using a minimum size of the luma codec unit generated by performing the binary division and a maximum size of the luma codec unit, the binary division may be determined in the intra-frame slice; determining information about a difference between the maximum size and the minimum size of a luma codec unit generated by performing the three-way division on the intra-frame slice by using a minimum size of the luma codec unit generated by performing the three-way division and a maximum size of the luma codec unit, the three-way division may be determined in the intra-frame slice information on the difference between the maximum size and the minimum size of the luma codec unit generated by performing the binary division and the maximum size of the luma codec unit, the binary division can be determined in the inter-frame strip, and information on the difference between the maximum size and the minimum size of the luma codec unit generated by performing the binary division and the maximum size of the luma codec unit is determined; and information on the difference between the maximum size and the minimum size of the luma codec unit generated by performing the three-way division and the three-way division can be determined in the inter-frame strip.

[0025] The video encoding method may further include: outputting information about the minimum size of the luma codec unit, information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra-frame slice, information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice, information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing ternary division on the intra-frame slice, and information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing ternary division on the inter-frame slice in the bitstream.

[0026] The video encoding method may further include: determining information about a difference between the maximum size and the minimum size of a chroma codec unit generated by performing binary partitioning and a maximum size of the chroma codec unit, which binary partitioning may be determined in an intra-frame slice; and determining information about a difference between the maximum size and the minimum size of a chroma codec unit generated by performing three-way partitioning and a maximum size of the chroma codec unit, which three-way partitioning may be determined in an intra-frame slice.

[0027] The video encoding method may further include: encoding information indicating whether the luma codec unit and the chroma codec unit are determined separately, and outputting the encoded information in a bitstream. The video encoding method may further include: when the luma codec unit and the chroma codec unit are determined separately, encoding information about a difference between a maximum size and a minimum size of a chroma codec unit generated by performing binary division on an intra-frame slice and information about a difference between a maximum size and a minimum size of a chroma codec unit generated by performing ternary division on an intra-frame slice, and outputting the encoded information in a bitstream.

[0028] The video encoding method may further include: when the first codec unit in the codec unit is not adjacent to the outer boundary of the image, encoding the partition type information of the first codec unit, and outputting the encoded partition type information in the bitstream. The video encoding method may further include: dividing the height or width of the first codec unit in half, determining two second codec units, each of which has a depth increased by 1 compared to the depth of the first codec unit, and encoding the partition type information of the first codec unit indicating the binary partition; and dividing the height or width of the first codec unit at a ratio of 1:2:1, determining a third codec unit and two fourth codec units, the depth of the third codec unit is increased by 1 compared to the depth of the first codec unit, and the depth of each fourth codec unit is increased by 2 compared to the depth of the first codec unit, and encoding the partition type information of the first codec unit indicating the trifurcated partition.

[0029] In order to overcome the above technical problems, the video encoding device proposed in the present invention includes: a processor, which is configured to determine a codec unit by dividing at least one of the height and the width of a maximum codec unit having a first size; based on whether the height or the width of a non-square first codec unit including an outer boundary of an image in the codec unit is greater than the maximum transformation size, determine whether it is allowed to generate two second codec units by dividing at least one of the height and the width of the first codec unit; and encode the second codec unit generated from the first codec unit by transforming the second codec unit using the maximum transformation size; and a memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] FIG1 shows a schematic block diagram of an image decoding apparatus according to an embodiment.

[0031] FIG. 2 shows a flowchart of an image decoding method according to an embodiment.

[0032] FIG. 3 illustrates a process of determining at least one codec unit by dividing a current codec unit, performed by an image decoding apparatus according to an embodiment.

[0033] FIG. 4 illustrates a process of determining at least one codec unit by dividing a non-square codec unit, performed by an image decoding apparatus according to an embodiment.

[0034] FIG. 5 illustrates a process of dividing a codec unit based on at least one of block shape information and division shape pattern information, performed by an image decoding device according to an embodiment.

[0035] FIG. 6 illustrates a method for determining a preset codec unit from an odd number of codec units, performed by an image decoding apparatus according to an embodiment.

[0036] FIG. 7 illustrates an order in which a plurality of codec units are processed when an image decoding apparatus determines a plurality of codec units by dividing a current codec unit according to an embodiment.

[0037] FIG. 8 illustrates a process of determining that a current codec unit is to be divided into an odd number of codec units, performed by an image decoding apparatus, when codec units cannot be processed in a preset order according to an embodiment.

[0038] FIG. 9 illustrates a process of determining at least one codec unit by dividing a first codec unit, performed by an image decoding apparatus according to an embodiment.

[0039] 10 shows that when a second codec unit having a non-square shape (determined by dividing the first codec unit by the image decoding apparatus) satisfies a preset condition, shapes into which the second codec unit may be divided are restricted according to an embodiment.

[0040] 11 illustrates a process of dividing a square codec unit performed by an image decoding device when division shape mode information indicates that the square codec unit is not divided into four square codec units according to an embodiment.

[0041] FIG. 12 illustrates that a processing order among a plurality of codec units may be changed depending on a process of dividing the codec units according to an embodiment.

[0042] 13 illustrates a process of determining a depth of a codec unit when a shape and a size of a codec unit are changed when a codec unit is recursively divided so that a plurality of codec units are determined according to an embodiment.

[0043] FIG. 14 illustrates a depth that may be determined based on a shape and a size of a codec unit, and a partial index (PID) for distinguishing a codec unit according to an embodiment.

[0044] FIG. 15 illustrates determining a plurality of coding units based on a plurality of preset data units included in a picture according to an embodiment.

[0045] FIG. 16 illustrates a processing block used as a criterion for determining an order of determining reference codec units included in a picture according to an embodiment.

[0046] FIG. 17 shows a block diagram of a video encoding apparatus according to an embodiment.

[0047] FIG. 18 shows a flowchart of a video encoding method according to an embodiment.

[0048] FIG. 19 shows a block diagram of a video decoding apparatus according to an embodiment.

[0049] FIG. 20 shows a flowchart of a video decoding method according to an embodiment.

[0050] FIG. 21 shows the relationship between a maximum codec unit, a pipeline data unit, and a block according to an embodiment.

[0051] FIG. 22 shows three shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0052] FIG. 23 shows five shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0053] FIG. 24 shows seven shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0054] FIG. 25 shows nine shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to another embodiment.

[0055] FIG. 26 shows two shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0056] FIG. 27 shows two shapes of combinations of pipeline data units that can be determined only through quadtree partitioning or binary partitioning of a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0057] 28 shows three shapes of combinations of pipeline data units that may be determined by quadtree partitioning, ternary partitioning, and binary partitioning of a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0058] FIG. 29 shows a function for determining whether a codec unit is located at a boundary line of a pipeline data unit according to an embodiment.

[0059] FIG. 30 shows a syntax signaled through a sequence parameter set (SPS) according to an embodiment. DETAILED DESCRIPTION

[0060] The advantages and features of the present invention and methods of achieving the same will be described more fully with reference to the accompanying drawings, in which embodiments of the present invention are shown. However, the present invention may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; on the contrary, these embodiments are provided so that the present invention will be thorough and complete and fully convey the concepts of the present invention to those of ordinary skill in the art.

[0061] The terms used herein will be briefly described and the disclosed embodiments will be described in detail.

[0062] Considering the functions of the terms used in the embodiments, the terms used in the embodiments are selected from the commonly used terms that are currently widely used. However, the terms may become different according to the intentions of ordinary technicians in the field, precedents, or the emergence of new technologies. In addition, in specific cases, the terms are freely selected by the applicant of the present invention, and the meanings of these terms will be described in detail in the corresponding parts of the detailed description. Therefore, the terms used in the embodiments are not only the designation of the terms, but are defined based on the meanings of the terms and contents in the entire embodiment.

[0063] It is to be understood that the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise.

[0064] It should be understood that when a specific part “includes” a specific component, the part does not exclude another component but may further include another component unless the context clearly states otherwise.

[0065] As used herein, the term "part," "module," or "unit" refers to a software or hardware component that performs a predefined function. However, the term "unit" is not limited to software or hardware. A "unit" may be configured in an addressable storage medium or configured to operate one or more processors. Thus, for example, a "unit" may include components, processes, functions, properties, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables such as software components, object-oriented software components, class components, and task components. The functions provided by components and "units" may be combined into fewer components and "units," or further separated into additional components and "units."

[0066] According to an embodiment of the present invention, a "unit" may include a processor and a memory. The term "processor" should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some embodiments, a "processor" may represent an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term "processor" may refer to a combination of processing devices, such as a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors and a DSP core, or a combination of any other such configuration.

[0067] The term "memory" should be interpreted broadly to include any electronic component capable of storing electronic information. The term "memory" may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), nonvolatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage devices, registers, etc. When the processor can read information from and / or write information to the memory, the memory is declared to be in electronic communication with the processor. Memory integrated into the processor is in electronic communication with the processor.

[0068] Hereinafter, “image” may indicate a still image of a video, or may indicate a dynamic image such as a moving image, that is, a video itself.

[0069] In the following, "sample" means data assigned to the sample position of the image, that is, data to be processed. For example, pixel values ​​in the spatial domain image and transform coefficients on the transform area can be samples. A unit including at least one such sample can be defined as a block.

[0070] Furthermore, in the specification, a 'current block' may mean a block of a maximum coding unit, a coding unit, a prediction unit, or a transform unit of a current image to be encoded or decoded.

[0071] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement these embodiments. In addition, in order to clearly describe the present invention, parts irrelevant to the description of the present invention will be omitted in the accompanying drawings.

[0072] Hereinafter, an image encoding device and an image decoding device as well as an image encoding method and an image decoding method according to an embodiment will be described with reference to FIGS. 1 to 16. According to an embodiment, a method for determining a data unit of an image will be described with reference to FIGS. 3 to 16, according to an embodiment, a video encoding / decoding method considering hardware design will be described with reference to FIGS. 17 to 20, according to an embodiment, various combinations of pipeline data units will be described with reference to FIGS. 21 to 28, a method for determining whether a coding unit is outside a pipeline boundary line will be described with reference to FIG. 29, and according to an embodiment, a method for signaling information about the maximum size and minimum size of a block allowed based on a partitioning mode by a video encoding device 1700 and a video decoding device 1900 will be described with reference to FIG. 30.

[0073] Hereinafter, a method and apparatus for adaptively selecting a context model based on various shapes of a coding unit according to an embodiment of the present disclosure will be described with reference to FIG. 1 and FIG. 2 .

[0074] FIG1 shows a schematic block diagram of an image decoding apparatus according to an embodiment.

[0075] The image decoding apparatus 100 may include a receiver 110 and a decoder 120. The receiver 110 and the decoder 120 may include at least one processor. In addition, the receiver 110 and the decoder 120 may include a memory storing instructions to be executed by the at least one processor.

[0076] The receiver 110 may receive a bitstream. The bitstream includes information of an image encoded by the image encoding device 2200 described below. In addition, the bitstream may be transmitted from the image encoding device 2200. The image encoding device 2200 and the image decoding device 100 may be connected by wire or wirelessly, and the receiver 110 may receive the bitstream by wire or wirelessly. The receiver 110 may receive the bitstream from a storage medium such as an optical medium or a hard disk. The decoder 120 may reconstruct the image based on information obtained from the received bitstream. The decoder 120 may obtain syntax elements for reconstructing the image from the bitstream. The decoder 120 may reconstruct the image based on the syntax elements.

[0077] The operation of the image decoding device 100 will be described in detail with reference to FIG. 2 .

[0078] FIG. 2 shows a flowchart of an image decoding method according to an embodiment.

[0079] According to an embodiment of the present invention, the receiver 110 receives a bit stream.

[0080] The image decoding device 100 obtains a bin (binary bit) string corresponding to the division shape pattern of the codec unit from the bit stream (operation 210). The image decoding device 100 determines the division rule of the codec unit (operation 220). In addition, the image decoding device 100 divides the codec unit into a plurality of codec units based on at least one of the bin strings corresponding to the division shape pattern and the division rule (operation 230). The image decoding device 100 may determine the allowable first range of the size of the codec unit according to the aspect ratio of the codec unit in order to determine the division rule. The image decoding device 100 may determine the allowable second range of the size of the codec unit according to the division shape pattern of the codec unit in order to determine the division rule.

[0081] Next, the division of the encoding and decoding units will be described in detail according to an embodiment of the present invention.

[0082] First, a picture can be divided into one or more slices or one or more tiles. A slice or a tile can be a sequence of one or more maximum codec units (coding tree units (CTUs)). Compared with the maximum codec unit (CTU), there is a maximum coding block (coding tree block (CTB)) in concept.

[0083] A maximum coding block (CTB) represents an NxN block (where N is an integer) consisting of NxN samples. Each color component can be divided into one or more maximum coding blocks.

[0084] When a picture has three sample arrays (sample arrays of Y, Cr, and Cb components), the maximum codec unit (CTU) includes a maximum coding block of luma samples, two corresponding maximum coding blocks of chroma samples, and a syntax structure for encoding luma samples and chroma samples. When the picture is a monochrome picture, the maximum codec unit includes a maximum coding block of monochrome samples and a syntax structure for encoding monochrome samples. When the picture is a picture encoded in a color plane separated according to color components, the maximum codec unit includes a syntax structure for encoding the picture and the samples of the picture.

[0085] A maximum coding block (CTB) can be divided into MxN coding blocks, including MxN samples (M and N are integers).

[0086] When a picture has sample arrays of Y, Cr, and Cb components, a codec unit (CU) includes a coding block of luma samples, two corresponding coding blocks of chroma samples, and a syntax structure for encoding luma samples and chroma samples. When a picture is a monochrome picture, a codec unit includes a coding block of monochrome samples and a syntax structure for encoding monochrome samples. When a picture is coded in color planes separated according to color components, a codec unit includes a syntax structure for encoding the picture and samples of the picture.

[0087] As described above, the maximum coding block and the maximum codec unit are conceptually distinguished, and the coding block and the codec unit are conceptually distinguished. That is, the (maximum) codec unit refers to a data structure including a (maximum) coding block (including corresponding samples) and a syntax structure corresponding to the (maximum) coding block. However, as those skilled in the art understand, a (maximum) codec unit or a (maximum) coding block refers to a block of a preset size including a preset number of samples, a maximum coding block, and a maximum codec unit, or, unless otherwise specified, the coding block and the codec unit are mentioned in the following specification without distinction.

[0088] The image may be divided into a maximum codec unit (CTU). The size of the maximum codec unit may be determined based on information obtained from the bitstream. The maximum codec unit may be in the shape of a square having the same size, but is not limited thereto. However, the embodiment is not limited thereto.

[0089] For example, information about the maximum size of the luma coding block may be obtained from the bitstream. For example, the maximum size of the luma coding block indicated by the information about the maximum size of the luma coding block may be one of 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, and 256x256.

[0090] For example, information about the luminance block size difference and the maximum size of the luminance coding block that can be divided into two can be obtained from the bitstream. The information about the luminance block size difference may refer to the size difference between the luminance maximum codec unit and the maximum luminance coding block that can be divided into two. Therefore, when the information about the maximum size of the luminance coding block that can be divided into two and the information about the luminance block size difference obtained from the bitstream are combined with each other, the size of the luminance maximum codec unit can be determined. The size of the chrominance maximum codec unit can be determined by using the size of the luminance maximum codec unit. For example, when the Y:Cb:Cr ratio according to the color format is 4:2:0, the size of the chrominance block may be half the size of the luminance block, and the size of the chrominance maximum codec unit may be half the size of the luminance maximum codec unit.

[0091] According to an embodiment, since information about the maximum size of a binary-dividable luminance coding block is obtained from a bitstream, the maximum size of a binary-dividable luminance coding block can be variably determined. In contrast, the maximum size of a trifurcated luminance coding block can be fixed. For example, the maximum size of a trifurcated luminance coding block in an I picture can be 32x32, while the maximum size of a trifurcated luminance coding block in a P picture or a B picture can be 64x64.

[0092] In addition, the maximum codec unit can be hierarchically divided into codec units based on the division shape pattern information obtained from the bitstream. At least one of information indicating whether to perform quad division, information indicating whether to perform multi-division, division direction information, and division type information can be obtained from the bitstream as the division shape pattern information.

[0093] For example, the information indicating whether to perform quad splitting may indicate whether the current coding unit is quad splitting (QUAD_SPLIT).

[0094] When the current codec unit is not quad-split, the information indicating whether to perform multi-split may indicate whether the current codec unit is no longer split (NO_SPLIT) or is binary / tri-split.

[0095] When the current codec unit is binary-divided or ternary-divided, the division direction information indicates that the current codec unit is divided in one of a horizontal direction and a vertical direction.

[0096] When the current codec unit is split in the horizontal direction or the vertical direction, the split type information indicates whether the current codec unit is binary split or ternary split.

[0097] The split mode of the current codec unit may be determined according to the split direction information and the split type information. The split mode when the current codec unit is split into two branches in the horizontal direction may be determined as a binary horizontal split mode (SPLIT_BT_HOR), the split mode when the current codec unit is split into three branches in the horizontal direction may be determined as a three-branch horizontal split mode (SPLIT_TT_HOR), the split mode when the current codec unit is split into two branches in the vertical direction may be determined as a binary vertical split mode (SPLIT_BT_VER), and the split mode when the current codec unit is split into three branches in the vertical direction may be determined as a three-branch vertical split mode (SPLIT_TT_VER).

[0098] The image decoding device 100 can obtain the division shape mode information from the bit stream and from a bin string. The form of the bit stream received by the image decoding device 100 may include a fixed-length binary code, a unary code, a truncated unary code, a predetermined binary code, etc. The bin string is information in the form of a binary number. The bin string may include at least one bit. The image decoding device 100 can obtain the division shape mode information corresponding to the bin string based on the division rule. The image decoding device 100 can determine whether to divide the codec unit into four forks, whether to divide the codec unit, the division direction, and the division type based on a bin string.

[0099] The codec unit may be less than or equal to the maximum codec unit. For example, because the maximum codec unit is a codec unit with a maximum size, the maximum codec unit is one of the codec units. When the division shape pattern information about the maximum codec unit indicates that the division is not performed, the codec unit determined in the maximum codec unit has the same size as the maximum codec unit. When the division shape pattern information about the maximum codec unit indicates that the division is performed, the maximum codec unit may be divided into codec units. In addition, when the division shape pattern information about the codec unit indicates that the division is performed, the codec unit may be divided into smaller codec units. However, the division of the image is not limited to this, and the maximum codec unit and the codec unit may not be distinguished. The division of the codec unit will be described in detail with reference to Figures 3 to 16.

[0100] In addition, one or more prediction blocks for prediction may be determined from the codec unit. The prediction block may be the same as or smaller than the codec unit. In addition, one or more transform blocks for transformation may be determined from the codec unit. The transform block may be the same as or smaller than the codec unit.

[0101] The shapes and sizes of the transform block and the prediction block may be unrelated to each other.

[0102] In another embodiment, prediction may be performed by using the codec unit as a prediction block. In addition, transformation may be performed by using the codec unit as a transformation block.

[0103] The division of the codec unit will be described in detail with reference to FIGS. 3 to 16. The current block and the adjacent block of the present invention may indicate one of the maximum codec unit, the codec unit, the prediction block, and the transform block. In addition, the current block or the current codec unit may be a block currently decoded or encoded or a block currently divided. The adjacent block may be a block reconstructed earlier than the current block. The adjacent block may be adjacent to the current block in the spatial domain or the temporal domain. The adjacent block may be located at any one of the lower left side, the upper left side, the upper side, the upper right side, the right side, and the lower right side of the current block.

[0104] FIG. 3 illustrates a process of determining at least one codec unit by dividing a current codec unit, performed by an image decoding apparatus according to an embodiment.

[0105] The block shape may include 4Nx4N, 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N. Here, N may be a positive integer. The block shape information is information indicating at least one of the shape, direction, aspect ratio, or size of the coding unit.

[0106] The shape of the codec unit may include a square and a non-square. When the width and height of the codec unit are the same (ie, when the block shape of the codec unit is 4Nx4N), the image decoding device 100 may determine the block shape information of the codec unit as a square. The image decoding device 100 may determine the shape of the codec unit as a non-square.

[0107] When the lengths of the width and height of the codec unit are different from each other (i.e., when the block shape of the codec unit is 4Nx2N, 2Nx4N, 4NxN, Nx4N, 32NxN, Nx32N, 16NxN, Nx16N, 8NxN, or Nx8N), the image decoding device 100 may determine the block shape information of the codec unit as a non-square. When the shape of the codec unit is a non-square, the image decoding device 100 may determine the aspect ratio in the block shape information of the codec unit as at least one of 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. In addition, the image decoding device 100 may determine whether the codec unit is in the horizontal direction or the vertical direction based on the length of the width and the length of the height of the codec unit. In addition, the image decoding device 100 may determine the size of the codec unit based on at least one of the length of the width, the length of the height, or the area of ​​the codec unit.

[0108] According to an embodiment, the image decoding device 100 may determine the shape of the codec unit by using the block shape information, and may determine the division shape of the codec unit by using the division shape mode information. That is, the division method of the codec unit indicated by the division shape mode information may be determined based on the block shape indicated by the block shape information used by the image decoding device 100.

[0109] The image decoding device 100 may obtain the division shape pattern information from the bitstream. However, the embodiment is not limited thereto, and the image decoding device 100 and the image encoding device 2200 may determine the pre-agreed division shape pattern information based on the block shape information. The image decoding device 100 may determine the pre-agreed division shape pattern information about the maximum codec unit or the minimum codec unit. For example, the image decoding device 100 may determine that the division shape pattern information about the maximum codec unit is a quadrilateral division. In addition, the image decoding device 100 may determine the division shape pattern information about the minimum codec unit as "no division is performed". Specifically, the image decoding device 100 may determine that the size of the maximum codec unit is 256x256. The image decoding device 100 may determine the pre-agreed division shape pattern information as a quadrilateral division. The quadrilateral division is a division shape pattern in which the width and height of the codec unit are equally divided. The image decoding device 100 may obtain a codec unit of 128x128 size from a maximum codec unit of 256x256 size based on the division shape pattern information. Furthermore, the image decoding device 100 may determine the size of the minimum decoding unit to be 4×4. The image decoding device 100 may obtain the division shape pattern information indicating “do not perform division” with respect to the minimum decoding unit.

[0110] According to an embodiment, the image decoding device 100 may use block shape information indicating that the current codec unit has a square shape. For example, the image decoding device 100 may determine whether to not divide the square codec unit, whether to divide the square codec unit vertically, whether to divide the square codec unit horizontally, or whether to divide the square codec unit into four codec units based on the division shape mode information. Referring to FIG. 3, when the block shape information of the current codec unit 300 indicates a square, the decoder 120 may determine that the codec unit 310a having the same size as the current codec unit 300 is not divided based on the division shape mode information indicating that division is not performed, or may determine that the codec unit 310b, 310c, 310d, 310e, or 310f is divided based on the division shape mode information indicating a preset division method.

[0111] 3 , according to an embodiment, the image decoding device 100 may determine two codec units 310b obtained by dividing the current codec unit 300 in the vertical direction based on the division shape pattern information indicating that the division is performed in the vertical direction. The image decoding device 100 may determine two codec units 310c obtained by dividing the current codec unit 300 in the horizontal direction based on the division shape pattern information indicating that the division is performed in the horizontal direction. The image decoding device 100 may determine four codec units 310d obtained by dividing the current codec unit 300 in the vertical direction and the horizontal direction based on the division shape pattern information indicating that the division is performed in the vertical and horizontal directions. According to an embodiment, the image decoding device 100 may determine three codec units 310e obtained by dividing the current codec unit 300 in the vertical direction based on the division shape pattern information indicating that the three-branch division is performed in the vertical direction. The image decoding device 100 may determine three codec units 310f obtained by dividing the current codec unit 300 in the horizontal direction based on the division shape pattern information indicating that the three-branch division is performed in the horizontal direction. However, the division shapes into which the square codec unit may be divided are not limited to the above method, and the division shape pattern information may indicate various methods. The preset division shapes into which the square codec unit may be divided will be described in detail below in conjunction with various embodiments.

[0112] FIG. 4 illustrates a process of determining at least one codec unit by dividing a non-square codec unit, performed by an image decoding apparatus according to an embodiment.

[0113] According to an embodiment, the image decoding device 100 may use block shape information indicating that the current codec unit has a non-square shape. The image decoding device 100 may determine whether to divide the non-square current codec unit based on the division shape mode information, or determine whether to divide the non-square current codec unit by using a preset division method. Referring to FIG. 4, when the block shape information of the current codec unit 400 or 450 indicates a non-square shape, the image decoding device 100 may determine that the codec unit 410 or 460 having the same size as the current codec unit 400 or 450 is not divided based on the division shape mode information indicating that the division is not performed, or determine that the codec units 420a and 420b, 430a to 430c, 470a and 470b, or 480a to 480c are divided based on the division shape mode information indicating the preset division method. The preset division method for dividing the non-square codec unit will be described in detail below in conjunction with various embodiments.

[0114] According to an embodiment, the image decoding device 100 may determine the division shape into which the codec unit is to be divided by using the division shape pattern information, and in this case, the division shape pattern information may indicate the number of one or more codec units generated by dividing the codec unit. Referring to FIG. 4, when the division shape pattern information indicates that the current codec unit 400 or 450 is divided into two codec units, the image decoding device 100 may determine the two codec units 420a and 420b, or 470a and 470b included in the current codec unit 400 or 450 by dividing the current codec unit 400 or 450 based on the division shape pattern information.

[0115] According to an embodiment, when the image decoding device 100 divides the non-square current codec unit 400 or 450 based on the division shape pattern information, the image decoding device 100 may divide the current codec unit in consideration of the position of the long side of the non-square current codec unit 400 or 450. For example, considering the shape of the current codec unit 400 or 450, the image decoding device 100 may determine a plurality of codec units by dividing the current codec unit 400 or 450 in a direction in which the long side of the current codec unit 400 or 450 is divided.

[0116] According to an embodiment, when the division shape mode information indicates that the codec unit is divided (tripartitioned) into an odd number of blocks, the image decoding device 100 may determine an odd number of codec units included in the current codec unit 400 or 450. For example, when the division shape mode information indicates that the current codec unit 400 or 450 is divided into three codec units, the image decoding device 100 may divide the current codec unit 400 or 450 into three codec units 430a, 430b, and 430c, or 480a, 480b, and 480c.

[0117] According to an embodiment, the aspect ratio of the current codec unit 400 or 450 may be 4:1 or 1:4. When the aspect ratio is 4:1, the block shape information may indicate the horizontal direction because the length of the width is longer than the length of the height. When the aspect ratio is 1:4, the block shape information may indicate the vertical direction because the length of the width is shorter than the length of the height. The image decoding device 100 may determine to divide the current codec unit into an odd number of blocks based on the division shape mode information. In addition, the image decoding device 100 may determine the division direction of the current codec unit 400 or 450 based on the block shape information of the current codec unit 400 or 450. For example, when the current codec unit 400 is in the vertical direction, the image decoding device 100 may determine the codec units 430a to 430c by dividing the current codec unit 400 in the horizontal direction. In addition, when the current codec unit 450 is in the horizontal direction, the image decoding device 100 may determine the codec units 480a to 480c by dividing the current codec unit 450 in the vertical direction.

[0118] According to an embodiment, the image decoding device 100 may determine an odd number of codec units included in the current codec unit 400 or 450, and not all determined codec units may have the same size. For example, the size of a preset codec unit 430b or 480b from among the determined odd number of codec units 430a, 430b, and 430c or 480a, 480b, and 480c may be different from the size of other codec units 430a and 430c or 480a and 480c. That is, the codec unit determined by dividing the current codec unit 400 or 450 may have a plurality of sizes, and in some cases, all odd number of codec units 430a, 430b, and 430c or 480a, 480b, and 480c may have different sizes.

[0119] According to an embodiment, when the division shape mode information indicates that the codec unit is divided into an odd number of blocks, the image decoding device 100 may determine an odd number of codec units included in the current codec unit 400 or 450, and may apply a preset restriction to at least one of the odd number of codec units generated by dividing the current codec unit 400 or 450. Referring to FIG4 , the image decoding device 100 may set a decoding process for a codec unit 430b or 480b located in the middle of three codec units 430a, 430b, and 430c or 480a, 480b, and 480c generated as the current codec unit 400 or 450 is divided to be different from the decoding processes of the other codec units 430a and 430c or 480a and 480c. For example, unlike the other codec units 430a and 430c or 480a and 480c, the image decoding device 100 may restrict the codec unit 430b or 480b located at the center position from being divided any more or being divided only a preset number of times.

[0120] FIG. 5 illustrates a process of dividing a codec unit based on at least one of block shape information and division shape pattern information, performed by an image decoding device according to an embodiment.

[0121] According to an embodiment, the image decoding device 100 may determine whether to divide the square first codec unit 500 into codec units based on at least one of the block shape information and the division shape mode information. According to an embodiment, when the division shape mode information indicates that the first codec unit 500 is divided in the horizontal direction, the image decoding device 100 may determine the second codec unit 510 by dividing the first codec unit 500 in the horizontal direction. The first codec unit, the second codec unit, and the third codec unit used according to the embodiment are terms for understanding the relationship before and after dividing the codec unit. For example, the second codec unit may be determined by dividing the first codec unit, and the third codec unit may be determined by dividing the second codec unit. It should be understood that the relationship between the first codec unit, the second codec unit, and the third codec unit follows the above description.

[0122] According to an embodiment, the image decoding device 100 may determine whether to divide the determined second codec unit 510 into codec units based on the division shape mode information. Referring to FIG5, the image decoding device 100 may divide the non-square second codec unit 510 determined by dividing the first codec unit 500 into one or more third codec units 520a, 520b, 520c, and 520d based on the division shape mode information and at least one of the division shape mode information, or may not divide the non-square second codec unit 510. The image decoding device 100 may obtain the division shape mode information, and may obtain a plurality of second codec units (e.g., 510) of different shapes by dividing the first codec unit 500 based on the obtained division shape mode information, and may divide the second codec unit 510 by using the division method of the first codec unit 500 based on the division shape mode information. According to an embodiment, when the first codec unit 500 is divided into the second codec unit 510 based on the division shape pattern information of the first codec unit 500, the second codec unit 510 may also be divided into a third codec unit (e.g., 520a, or 520b, 520c, and 520d). That is, the codec unit may be recursively divided based on the division shape pattern information of each codec unit. Therefore, a square codec unit may be determined by dividing a non-square codec unit, and a non-square codec unit may be determined by recursively dividing a square codec unit.

[0123] 5, the preset codec units (e.g., codec units located at the center or square codec units) in the odd number of third codec units 520b, 520c, and 520d determined by dividing the non-square second codec unit 510 may be recursively divided. According to an embodiment, the square third codec unit 520c in the odd number of third codec units 520b, 520c, and 520d may be divided into a plurality of fourth codec units in the horizontal direction. The non-square fourth codec unit 530b or 530d in the plurality of fourth codec units 530a, 530b, 530c, and 530d may be re-divided into a plurality of codec units. For example, the non-square fourth codec unit 530b or 530d may be re-divided into an odd number of codec units. A method for recursively dividing a codec unit will be described below in conjunction with various embodiments.

[0124] According to an embodiment, the image decoding device 100 may divide each of the third codec units 520a or 520b, 520c, and 520d into codec units based on the division shape pattern information. In addition, the image decoding device 100 may determine not to divide the second codec unit 510 based on the division shape pattern information. According to an embodiment, the image decoding device 100 may divide the non-square second codec unit 510 into an odd number of third codec units 520b, 520c, and 520d. The image decoding device 100 may impose a preset restriction on a preset third codec unit among the odd number of third codec units 520b, 520c, and 520d. For example, the image decoding device 100 may limit the third codec unit 520c located at the center position among the odd number of third codec units 520b, 520c, and 520d to no longer be divided or to be divided a settable number of times.

[0125] 5, the image decoding device 100 may limit the third codec unit 520c located at the center position among the odd number of third codec units 520b, 520c, and 520d included in the non-square second codec unit 510 to no longer be divided, to be divided by using a preset division method (e.g., to be divided only into four codec units or to be divided using the division method of the second codec unit 510), or to be divided only a preset number of times (e.g., to be divided only n times (where n>0)). However, the limitation on the third codec unit 520c at the center position is not limited to the above example, and may include various limitations on decoding the third codec unit 520c located at the center position that are different from the other third codec units 520b and 520d.

[0126] According to an embodiment, the image decoding apparatus 100 may obtain division shape pattern information for dividing the current codec unit from a preset position in the current codec unit.

[0127] FIG. 6 illustrates a method for determining a preset codec unit from an odd number of codec units, performed by an image decoding apparatus according to an embodiment.

[0128] 6, the division shape mode information of the current codec unit 600 or 650 may be obtained from a sample at a preset position (e.g., a sample 640 or 690 at a center position) among a plurality of samples included in the current codec unit 600 or 650. However, the preset position where at least one piece of division shape mode information may be obtained from the current codec unit 600 is not limited to the center position in FIG. 6, and may include various positions (e.g., top, bottom, left, right, upper left, lower left, upper right, lower right position, etc.) included in the current codec unit 600. The image decoding device 100 may obtain the division shape mode information from the preset position, and may determine whether to divide the current codec unit into codec units of various shapes and various sizes.

[0129] According to an embodiment, when the current codec unit is divided into a preset number of codec units, the image decoding apparatus 100 may select one of the codec units. Various methods may be used to select one of the plurality of codec units, as will be described below in conjunction with various embodiments.

[0130] According to an embodiment, the image decoding apparatus 100 may divide a current codec unit into a plurality of codec units, and may determine a codec unit at a preset position.

[0131] According to an embodiment, the image decoding device 100 may determine a codec unit located at a center position among the odd codec units using information indicating the positions of the odd codec units. Referring to FIG6 , the image decoding device 100 may determine the odd codec units 620a, 620b, and 620c or the odd codec units 660a, 660b, and 660c by dividing the current codec unit 600 or the current codec unit 650. The image decoding device 100 may determine the intermediate codec unit 620b or the intermediate codec unit 660b by using information about the positions of the odd codec units 620a, 620b, and 620c or the positions of the odd codec units 660a, 660b, and 660c. For example, the image decoding device 100 can determine the codec unit 620b at the center position by determining the positions of the codec units 620a, 620b, and 620c based on the information indicating the positions of the preset samples included in the codec units 620a, 620b, and 620c. Specifically, the image decoding device 100 can determine the codec unit 620b at the center position by determining the positions of the codec units 620a, 620b, and 620c based on the information indicating the positions of the upper left samples 630a, 630b, and 630c of the codec units 620a, 620b, and 620c.

[0132] According to an embodiment, the information indicating the positions of the upper left corner sample points 630a, 630b, and 630c respectively included in the codec units 620a, 620b, and 620c may include information about the positions or coordinates of the codec units 620a, 620b, and 620c in the picture. According to an embodiment, the information indicating the positions of the upper left sample points 630a, 630b, and 630c respectively included in the codec units 620a, 620b, and 620c may include information indicating the width or height of the codec units 620a, 620b, and 620c included in the current codec unit 600, and the width or height may correspond to information indicating the difference between the coordinates of the codec units 620a, 620b, and 620c in the picture. That is, the image decoding device 100 can determine the codec unit 620b at the center position by directly using information about the positions or coordinates of the codec units 620a, 620b and 620c in the picture, or by using information about the width or height of the codec unit corresponding to the difference between the coordinates.

[0133] According to an embodiment, the information indicating the position of the upper left sample point 630a of the upper codec unit 620a may include coordinates (xa, ya), the information indicating the position of the upper left sample point 630b of the intermediate codec unit 620b may include coordinates (xb, yb), and the information indicating the position of the upper left sample point 630c of the lower codec unit 620c may include coordinates (xc, yc). The image decoding device 100 may determine the intermediate codec unit 620b by using the coordinates of the upper left sample points 630a, 630b, and 630c respectively included in the codec units 620a, 620b, and 620c. For example, when the coordinates of the upper left sample points 630a, 630b, and 630c are sorted in ascending or descending order, the codec unit 620b at the center position including the coordinates (xb, yb) of the sample point 630b may be determined as the codec unit at the center position among the codec units 620a, 620b, and 620c determined by dividing the current codec unit 600. However, the coordinates indicating the positions of the upper left sample points 630a, 630b, and 630c may include coordinates indicating absolute positions in the picture, or coordinates (dxb, dyb) indicating a relative position of the upper left sample point 630b of the middle codec unit 620b, and coordinates (dxc, dyc) indicating a relative position of the upper left sample point 630c of the lower codec unit 620c may be used with respect to the position of the upper left sample point 630a of the upper codec unit 620a. The method of determining the codec unit at a preset position by using the coordinates of the samples included in the codec unit as information indicating the positions of the samples is not limited to the above method and may include various arithmetic methods capable of using the coordinates of the samples.

[0134] According to an embodiment, the image decoding device 100 may divide the current codec unit 600 into a plurality of codec units 620a, 620b, and 620c, and may select one of the codec units 620a, 620b, and 620c based on a preset criterion. For example, the image decoding device 100 may select a codec unit 620b having a size different from that of other codec units from the codec units 620a, 620b, and 620c.

[0135] According to an embodiment, the image decoding device 100 may determine the width or height of each of the codec units 620a, 620b, and 620c by using coordinates (xa, ya), which are information indicating the position of the upper left sample point 630a of the upper codec unit 620a, coordinates (xb, yb), and coordinates (xc, yc), which are information indicating the position of the upper left sample point 630b of the middle codec unit 620b, and coordinates (xc, yc), which are information indicating the position of the upper left sample point 630c of the lower codec unit 620c. The image decoding device 100 may determine the respective sizes of the codec units 620a, 620b, and 620c by using coordinates (xa, ya), (xb, yb), and (xc, yc) indicating the codec units 620a, 620b, and 620c. According to an embodiment, the image decoding device 100 may determine the width of the upper codec unit 620a as the width of the current codec unit 600. The image decoding device 100 may determine the height of the upper codec unit 620a as yb-ya. According to an embodiment, the image decoding device 100 may determine the width of the middle codec unit 620b as the width of the current codec unit 600. The image decoding device 100 may determine the height of the middle codec unit 620b as yc-yb. According to an embodiment, the image decoding device 100 may determine the width or height of the lower codec unit 620c by using the width or height of the current codec unit 600 or the width or height of the upper codec unit 620a and the middle codec unit 620b. The image decoding device 100 may determine a codec unit having a size different from other codec units based on the determined widths and heights of the codec units 620a to 620c. 6, the image decoding device 100 may determine the middle codec unit 620b having a size different from that of the upper codec unit 620a and the lower codec unit 620c as the codec unit at the preset position. However, the above-mentioned method of determining the codec unit having a size different from that of other codec units performed by the image decoding device 100 corresponds only to an example of determining the codec unit at the preset position by using the size of the codec unit determined based on the coordinates of the sample points, and therefore, various methods of determining the codec unit at the preset position by comparing the size of the codec unit determined based on the coordinates of the preset sample points may be used.

[0136] The image decoding device 100 may determine the width or height of each of the codec units 660a, 660b, and 660c by using coordinates (xd, yd), (xe, ye), and (xf, yf), the coordinates (xd, yd) being information indicating the position of the upper left sample point 670a of the left codec unit 660a, the coordinates (xe, ye) being information indicating the position of the upper left sample point 670b of the middle codec unit 660b, and the coordinates (xf, yf) being information indicating the position of the upper left sample point 670c of the right codec unit 660c. The image decoding device 100 may determine the respective sizes of the codec units 660a, 660b, and 660c by using the coordinates (xd, yd), (xe, ye), and (xf, yf) indicating the positions of the codec units 660a, 660b, and 660c.

[0137] According to an embodiment, the image decoding device 100 may determine the width of the left codec unit 660a as xe-xd. The image decoding device 100 may determine the height of the left codec unit 660a as the height of the current codec unit 650. According to an embodiment, the image decoding device 100 may determine the width of the middle codec unit 660b as xf-xe. The image decoding device 100 may determine the height of the middle codec unit 660b as the height of the current codec unit 650. According to an embodiment, the image decoding device 100 may determine the width or height of the right codec unit 660c by using the width or height of the current codec unit 650 or the width or height of the left codec unit 660a and the middle codec unit 660b. The image decoding device 100 may determine a codec unit having a size different from other codec units based on the determined width or height of the codec units 660a to 660c. 6, the image decoding device 100 may determine the middle codec unit 660b having a size different from the left codec unit 660a and the right codec unit 660c as the codec unit at the preset position. However, the above-mentioned method of determining the codec unit having a size different from the sizes of other codec units performed by the image decoding device 100 corresponds only to an example of determining the codec unit at the preset position by using the size of the codec unit determined based on the coordinates of the sample points, and therefore, various methods of determining the codec unit at the preset position by comparing the size of the codec unit determined based on the coordinates of the preset sample points may be used.

[0138] However, the position of the sample points used to determine the position of the codec unit is not limited to the above-mentioned upper left position, and information on arbitrary positions of the samples included in the codec unit may be used.

[0139] According to an embodiment, the image decoding device 100 may select a codec unit at a preset position from an odd number of codec units determined by dividing the current codec unit, taking into account the shape of the current codec unit. For example, when the current codec unit has a non-square shape with a width greater than a height, the image decoding device 100 may determine a codec unit at a preset position in the horizontal direction. That is, the image decoding device 100 may determine one of the codec units at different positions in the horizontal direction and impose restrictions on the codec unit. When the current codec unit has a non-square shape with a height greater than a width, the image decoding device 100 may determine a codec unit at a preset position in the vertical direction. That is, the image decoding device 100 may determine one of the codec units at different positions in the vertical direction and may impose restrictions on the codec unit.

[0140] According to an embodiment, the image decoding device 100 may use information indicating the corresponding positions of the even-numbered codec units to determine the codec unit at the preset position from the even-numbered codec units. The image decoding device 100 may determine the even-numbered codec units by dividing (binary division) the current codec unit, and may determine the codec unit at the preset position by using information about the positions of the even-numbered codec units. Operations related to this may correspond to operations for determining the codec unit at the preset position (e.g., the center position) among the odd-numbered codec units, which has been described in detail above with respect to FIG. 6, and therefore a detailed description thereof is not provided herein.

[0141] According to an embodiment, when a non-square current codec unit is divided into a plurality of codec units, a codec unit at a preset position among the plurality of codec units may be determined using preset information about the codec unit at the preset position in the division operation. For example, the image decoding apparatus 100 may determine a codec unit at a center position among the plurality of codec units determined by dividing the current codec unit using at least one of block shape information and division shape pattern information stored in samples included in the middle codec unit in the division operation.

[0142] 6, the image decoding device 100 may divide the current codec unit 600 into a plurality of codec units 620a, 620b, and 620c based on the division shape pattern information, and may determine the codec unit 620b at the center position among the plurality of codec units 620a, 620b, and 620c. In addition, the image decoding device 100 may determine the codec unit 620b at the center position in consideration of the position from which the division shape pattern information is obtained. That is, the division shape pattern information of the current codec unit 600 may be obtained from the sample point 640 at the center position of the current codec unit 600, and when the current codec unit 600 is divided into a plurality of codec units 620a, 620b, and 620c based on the division shape pattern information, the codec unit 620b including the sample point 640 may be determined as the codec unit at the center position. However, the information for determining the codec unit at the center position is not limited to the division shape pattern information, and various types of information may be used to determine the codec unit at the center position.

[0143] According to an embodiment, preset information for identifying a codec unit at a preset position may be obtained from preset samples included in the codec unit to be determined. Referring to FIG. 6 , the image decoding device 100 may use the division shape pattern information obtained from the samples at the preset position in the current codec unit 600 (e.g., the samples at the center position of the current codec unit 600) to determine the codec unit at the preset position (e.g., the codec unit at the center position of the multiple divided codec units) among the multiple codec units 620a, 620b, and 620c determined by dividing the current codec unit 600. That is, the image decoding device 100 may determine the samples at the preset position in consideration of the block shape of the current codec unit 600, determine the codec unit 620b including the samples from which the preset information (e.g., the division shape pattern information) can be obtained from the multiple codec units 620a, 620b, and 620c determined by dividing the current codec unit 600, and may impose a preset restriction on the codec unit 620b. 6, according to an embodiment, the image decoding device 100 may determine a sample 640 at the center position of the current codec unit 600 as a sample from which preset information may be obtained, and may impose preset restrictions on a codec unit 620b including the sample 640 in a decoding operation. However, the position of the sample from which the preset information may be obtained is not limited to the above-mentioned position, and may include any position of the sample included in the codec unit 620b to be determined for restriction.

[0144] According to an embodiment, the position of the sample from which the preset information can be obtained may be determined based on the shape of the current codec unit 600. According to an embodiment, the block shape information may indicate whether the current codec unit is in a square shape or a non-square shape, and the position of the sample from which the preset information can be obtained may be determined based on the shape. For example, the image decoding device 100 may determine the sample located on the boundary for dividing by half of at least one of the width and height of the current codec unit as the sample from which the preset information can be obtained by using at least one of the information about the width of the current codec unit and the information about the height of the current codec unit. As another example, when the block shape information of the current codec unit indicates a non-square shape, the image decoding device 100 may determine one of the samples including the boundary for dividing by half of the long side of the current codec unit as the sample from which the preset information can be obtained.

[0145] According to an embodiment, when the current codec unit is divided into a plurality of codec units, the image decoding device 100 may determine a codec unit at a preset position in the plurality of codec units using the division shape pattern information. According to an embodiment, the image decoding device 100 may obtain the division shape pattern information from the sample at the preset position in the codec unit, and divide the plurality of codec units generated by dividing the current codec unit by using the division shape pattern information (obtained from the sample at the preset position in each of the plurality of codec units). That is, the codec unit may be recursively divided based on the division shape pattern information obtained from the sample at the preset position in each codec unit. The operation of recursively dividing the codec unit has been described above with reference to FIG. 5, and thus a detailed description thereof is not provided herein.

[0146] According to an embodiment, the image decoding apparatus 100 may determine one or more codec units by dividing a current codec unit, and may determine an order of decoding the one or more codec units based on a preset block (eg, the current codec unit).

[0147] FIG. 7 illustrates an order in which a plurality of codec units are processed when an image decoding apparatus determines a plurality of codec units by dividing a current codec unit according to an embodiment.

[0148] According to an embodiment, based on the division shape pattern information, the image decoding device 100 can determine the second codec units 710a and 710b by dividing the first codec unit 700 in the vertical direction, determine the second codec units 730a and 730b by dividing the first codec unit 700 in the horizontal direction, or determine the second codec units 750a, 750b, 750c and 750d by dividing the first codec unit 700 in the vertical direction and the horizontal direction.

[0149] 7 , the image decoding apparatus 100 may determine to process the second codec units 710a and 710b determined by dividing the first codec unit 700 in the vertical direction in a horizontal direction order 710c. The image decoding apparatus 100 may determine to process the second codec units 730a and 730b determined by dividing the first codec unit 700 in the horizontal direction in a vertical direction order 730c. The image decoding apparatus 100 may determine the second codec units 750a to 750d determined by dividing the first codec unit 700 in the vertical direction and the horizontal direction according to a preset order (e.g., a raster scan order or a Z scan order 750e) of processing codec units in one row and then processing codec units in the next row.

[0150] According to an embodiment, the image decoding device 100 may recursively divide the codec unit. Referring to FIG. 7, the image decoding device 100 may determine a plurality of codec units 710a and 710b, 730a and 730b, or 750a to 750d by dividing the first codec unit 700, and recursively divide each of the determined plurality of codec units 710a, 710b, 730a, 730b, 750a, 750b, 750c, and 750d. The division method of the plurality of codec units 710a and 710b, 730a and 730b, or 750a to 750d may correspond to the division method of the first codec unit 700. In this way, each of the plurality of codec units 710a and 710b, 730a and 730b, or 750a to 750d may be independently divided into a plurality of codec units. 7 , the image decoding device 100 may determine second codec units 710 a and 710 b by dividing the first codec unit 700 in a vertical direction, and may determine to independently divide or not divide each of the second codec units 710 a and 710 b .

[0151] According to an embodiment, the image decoding device 100 may determine the third codec units 720a and 720b by dividing the left second codec unit 710a in the horizontal direction, and may not divide the right second codec unit 710b.

[0152] According to an embodiment, the processing order of the codec units may be determined based on the operation of dividing the codec units. In other words, the processing order of the divided codec units may be determined based on the processing order of the codec units immediately before being divided. The image decoding device 100 may determine the processing order of the third codec units 720a and 720b determined by dividing the left second codec unit 710a independently of the right second codec unit 710b. Since the third codec units 720a and 720b are determined by dividing the left second codec unit 710a in the horizontal direction, the third codec units 720a and 720b may be processed in a vertical direction order 720c. Since the left second codec unit 710a and the right second codec unit 710b are processed in a horizontal direction order 710c, the right second codec unit 710b may be processed after the third codec units 720a and 720b included in the left second codec unit 710a are processed in a vertical direction order 720c. The operation of determining the processing order of codec units based on the codec units before being divided is not limited to the above example, and codec units divided and determined into various shapes may be independently processed in a preset order using various methods.

[0153] FIG. 8 illustrates a process of determining that a current codec unit is to be divided into an odd number of codec units, performed by an image decoding apparatus, when codec units cannot be processed in a preset order according to an embodiment.

[0154] According to an embodiment, the image decoding device 100 may determine that the current codec unit is divided into an odd number of codec units based on the obtained division shape pattern information. Referring to FIG. 8 , the square first codec unit 800 may be divided into non-square second codec units 810a and 810b, and the second codec units 810a and 810b may be independently divided into third codec units 820a and 820b, and 820c, 820d, and 820e. According to an embodiment, the image decoding device 100 may determine a plurality of third codec units 820a and 820b by dividing the left second codec unit 810a in the horizontal direction, and may divide the right second codec unit 810b into an odd number of third codec units 820c to 820e.

[0155] According to an embodiment, the image decoding device 100 may determine whether to divide any codec unit into an odd number of codec units by determining whether the third codec units 820a and 820b and 820c to 820e can be processed in a preset order. Referring to FIG. 8, the image decoding device 100 may determine the third codec units 820a and 820b and 820c to 820e by recursively dividing the first codec unit 800. The image decoding device 100 may determine whether any one of the first codec unit 800, the second codec units 810a and 810b, or the third codec units 820a and 820b, and 820c to 820e is divided into an odd number of codec units based on at least one of the block shape information and the division shape mode information. For example, the codec unit located on the right side of the second codec units 810a and 810b may be divided into an odd number of third codec units 820c, 820d, and 820e. The processing order of the multiple codec units included in the first codec unit 800 can be a preset order (for example, a Z scan order 830), and the image decoding device 100 can determine whether the third codec units 820c, 820d and 820e determined by dividing the right second codec unit 810b into an odd number of codec units meet the conditions for processing in the preset order.

[0156] According to an embodiment, the image decoding device 100 may determine whether the third codec units 820a and 820b and 820c to 820e included in the first codec unit 800 satisfy a condition for processing in a preset order, and the condition relates to whether at least one of the width and height of the second codec units 810a and 810e is divided in half along the boundary of the third codec units 820a and 820b and 820c to 820e. For example, the third codec units 820a and 820b determined when the height of the non-square-shaped left second codec unit 810a is divided in half may satisfy the condition. It may be determined that the third codec units 820c to 820e do not satisfy the condition because the boundary of the third codec units 820c to 820e determined when the right second codec unit 810b is divided into three codec units cannot be divided by half of the width or height of the right second codec unit 810b. When the condition is satisfied as described above, the image decoding device 100 may determine the disconnection of the scanning order, and may determine to divide the right second codec unit 810b into an odd number of codec units based on the result of the determination. According to an embodiment, when the codec unit is divided into an odd number of codec units, the image decoding device 100 may impose a preset restriction on the codec unit at a preset position in the divided codec unit. The restriction or preset position has been described above in conjunction with various embodiments, so a detailed description thereof will not be provided here.

[0157] FIG. 9 illustrates a process of determining at least one codec unit by dividing a first codec unit, performed by an image decoding apparatus according to an embodiment.

[0158] According to an embodiment, the image decoding device 100 may divide the first codec unit 900 based on the division shape pattern information obtained by the receiver 110. The square first codec unit 900 may be divided into four square codec units, or may be divided into a plurality of non-square codec units. For example, referring to FIG. 9, when the first codec unit 900 has a square shape and the division shape pattern information indicates that the first codec unit 900 is divided into non-square codec units, the image decoding device 100 may divide the first codec unit 900 into a plurality of non-square codec units. Specifically, when the division shape pattern information indicates that an odd number of codec units is determined by dividing the first codec unit 900 in the horizontal direction or the vertical direction, the image decoding device 100 may divide the square first codec unit 900 into an odd number of codec units, for example, second codec units 910a, 910b and 910c determined by dividing the square first codec unit 900 in the vertical direction, or second codec units 920a, 920b and 920c determined by dividing the square first codec unit 900 in the horizontal direction.

[0159] According to an embodiment, the image decoding device 100 may determine whether the second codec units 910a, 910b, and 910c and 920a, 920b, and 920c included in the first codec unit 900 satisfy a condition for processing in a preset order, and the condition relates to whether at least one of the width and height of the first codec unit 900 is divided in half along the boundary of the second codec units 910a, 910b, and 910c and 920a, 920b, and 920c. Referring to FIG9, since the boundary of the second codec units 910a, 910b, and 910c determined by dividing the square first codec unit 900 in the vertical direction is not divided by half of the width of the first codec unit 900, it may be determined that the first codec unit 900 does not satisfy the condition sufficient for processing in the preset order. In addition, because the boundaries of the second codec units 920a, 920b, and 920c determined by dividing the square first codec unit 900 in the horizontal direction are not divided at half the height of the first codec unit 900, it can be determined that the first codec unit 900 does not meet the conditions sufficient for processing in a preset order. As described above, when the conditions are not met, the image decoding device 100 can decide to disconnect the scanning order, and can determine that the first codec unit 900 will be divided into an odd number of codec units based on the result of the decision. According to an embodiment, when the codec unit is divided into an odd number of codec units, the image decoding device 100 can impose preset restrictions on the codec units at preset positions in the divided codec units. The restrictions or preset positions have been described above in conjunction with various embodiments, and therefore a detailed description thereof will not be provided herein.

[0160] According to an embodiment, the image decoding device 100 may determine codec units of various shapes by dividing the first codec unit.

[0161] 9 , the image decoding device 100 may divide a square first codec unit 900 or a non-square first codec unit 930 or 950 into codec units of various shapes.

[0162] 10 shows that when a second codec unit having a non-square shape (determined by dividing the first codec unit by the image decoding apparatus) satisfies a preset condition, shapes into which the second codec unit may be divided are restricted according to an embodiment.

[0163] According to an embodiment, the image decoding device 100 may determine to divide the square first codec unit 1000 into non-square second codec units 1010a and 1010b or 1020a and 1020b based on the division shape pattern information obtained by the receiver 110. The second codec units 1010a and 1010b or 1020a and 1020b may be divided independently. In this way, the image decoding device 100 may determine whether to divide each of the second codec units 1010a and 1010b or 1020a and 1020b into a plurality of codec units based on the division shape pattern information of each of the second codec units 1010a and 1010b or 1020a and 1020b. According to an embodiment, the image decoding device 100 may determine the third codec units 1012a and 1012b by dividing the non-square left second codec unit 1010a determined by dividing the first codec unit 1000 in the vertical direction in the horizontal direction. However, when the left second codec unit 1010a is divided in the horizontal direction, the image decoding device 100 may restrict the right second codec unit 1010b from being divided in the horizontal direction in which the left second codec unit 1010a is divided. When the third codec units 1014a and 1014b are determined by dividing the right second codec unit 1010b in the same direction, since the left second codec unit 1010a and the right second codec unit 1010b are independently divided in the horizontal direction, the third codec units 1012a and 1012b or 1014a and 1014b may be determined. However, this case is the same as the case in which the image decoding device 100 divides the first codec unit 1000 into four square second codec units 1030a, 1030b, 1030c, and 1030d based on the division shape pattern information, and may be inefficient in image decoding.

[0164] According to an embodiment, the image decoding device 100 may determine the third codec units 1022a and 1022b or 1024a and 1024b by dividing in the vertical direction the non-square second codec unit 1020a or 1020b determined by dividing the first codec unit 1000 in the horizontal direction. However, due to the above reasons, when the second codec unit (e.g., the upper second codec unit 1020a) is divided in the vertical direction, the image decoding device 100 may restrict another second codec unit (e.g., the lower second codec unit 1020b) from being divided in the vertical direction in which the upper second codec unit 1020a is divided.

[0165] 11 illustrates a process of dividing a square codec unit performed by an image decoding device when division shape mode information indicates that the square codec unit is not divided into four square codec units according to an embodiment.

[0166] According to an embodiment, the image decoding device 100 may determine the second codec units 1110a and 1110b or 1120a and 1120b, etc. by dividing the first codec unit 1100 based on the division shape pattern information. The division shape pattern information may include information about various methods of dividing the codec unit, but the information about the various division methods may not include information for dividing the codec unit into four square codec units. According to such division shape pattern information, the image decoding device 100 may not divide the square first codec unit 1100 into four square second codec units 1130a, 1130b, 1130c, and 1130d. The image decoding device 100 may determine the non-square second codec units 1110a and 1110b or 1120a and 1120b, etc. based on the division shape pattern information.

[0167] According to an embodiment, the image decoding device 100 may independently divide the non-square second codec units 1110a and 1110b or 1120a and 1120b, etc. Each of the second codec units 1110a and 1110b or 1120a and 1120b, etc. may be recursively divided in a preset order, and the division method may correspond to the method of dividing the first codec unit 1100 based on the division shape pattern information.

[0168] For example, the image decoding device 100 can determine the square third codec units 1112a and 1112b by dividing the left second codec unit 1110a in the horizontal direction, and can determine the square third codec units 1114a and 1114b by dividing the right second codec unit 1110b in the horizontal direction. In addition, the image decoding device 100 can determine the square third codec units 1116a, 1116b, 1116c, and 1116d by dividing both the left second codec unit 1110a and the right second codec unit 1110b in the horizontal direction. In this case, a codec unit having the same shape as the four square second codec units 1130a, 1130b, 1130c, and 1130d divided from the first codec unit 1100 can be determined.

[0169] As another example, the image decoding device 100 may determine square third codec units 1122a and 1122b by dividing the upper second codec unit 1120a in the vertical direction, and may determine square third codec units 1124a and 1124b by dividing the lower second codec unit 1120b in the vertical direction. In addition, the image decoding device 100 may determine square third codec units 1126a, 1126b, 1126c, and 1126d by dividing both the upper second codec unit 1120a and the lower second codec unit 1120b in the vertical direction. In this case, a codec unit having the same shape as the four square second codec units 1130a, 1130b, 1130c, and 1130d divided from the first codec unit 1100 may be determined.

[0170] FIG. 12 illustrates that a processing order among a plurality of codec units may be changed depending on a process of dividing the codec units according to an embodiment.

[0171] According to an embodiment, the image decoding device 100 may divide the first codec unit 1200 based on the division shape pattern information. When the block shape indicates a square shape and the division shape pattern information indicates that the first codec unit 1200 is divided in at least one of the horizontal direction and the vertical direction, the image decoding device 100 may determine the second codec units 1210a and 1210b or 1220a and 1220b, etc. by dividing the first codec unit 1200. Referring to FIG. 12, the non-square second codec units 1210a and 1210b or 1220a and 1220b determined by dividing the first codec unit 1200 only in the horizontal direction or the vertical direction may be independently divided based on the division shape pattern information of each codec unit. For example, the image decoding device 100 may determine the third codec units 1216a, 1216b, 1216c, and 1216d by dividing the second codec units 1210a and 1210b generated by dividing the first codec unit 1200 in the vertical direction in the horizontal direction, and may determine the third codec units 1226a, 1226b, 1226c, and 1226d by dividing the second codec units 1220a and 1220b generated by dividing the first codec unit 1200 in the horizontal direction in the vertical direction. The operation of dividing the second codec units 1210a and 1210b or 1220a and 1220b has been described above with reference to FIG. 11, and thus a detailed description thereof is not provided herein.

[0172] According to an embodiment, the image decoding device 100 may process the codec units in a preset order. The operation of processing the codec units in a preset order has been described above with reference to FIG. 7, so a detailed description thereof is not provided herein. Referring to FIG. 12, the image decoding device 100 may determine four square third codec units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d by dividing the square first codec unit 1200. According to an embodiment, the image decoding device 100 may determine the processing order of the third codec units 1216a, 1216b, 1216c, and 1216d and 1226a, 1226b, 1226c, and 1226d based on the division shape into which the first codec unit 1200 is divided.

[0173] According to an embodiment, the image decoding device 100 can determine the third codec units 1216a, 1216b, 1216c and 1216d by dividing the second codec units 1210a and 1210b generated by dividing the first codec unit 1200 in the vertical direction in the horizontal direction, and can process the third codec units 1216a, 1216b, 1216c and 1216d in the processing order 1217 to first process the third codec units 1216a and 1216c included in the left second codec unit 1210a in the vertical direction, and then process the third codec units 1216b and 1216d included in the right second codec unit 1210b in the vertical direction.

[0174] According to an embodiment, the image decoding device 100 can determine the third codec units 1226a, 1226b, 1226c and 1226d by dividing the second codec units 1220a and 1220b generated by dividing the first codec unit 1200 in the horizontal direction in the vertical direction, and can process the third codec units 1226a, 1226b, 1226c and 1226d in the processing order 1227 to first process the third codec units 1226a and 1226b included in the upper second codec unit 1220a in the horizontal direction, and then process the third codec units 1226c and 1226d included in the lower second codec unit 1220b in the horizontal direction.

[0175] 12, square third codec units 1216a, 1216b, 1216c and 1216d and 1226a, 1226b, 1226c and 1226d can be determined by dividing the second codec units 1210a and 1210b and 1220a and 1220b, respectively. Although the second codec units 1210a and 1210b are determined by dividing the first codec unit 1200 in the vertical direction, unlike the second codec units 1220a and 1220b determined by dividing the first codec unit 1200 in the horizontal direction, the third codec units 1216a, 1216b, 1216c and 1216d and 1226a, 1226b, 1226c and 1226d divided therefrom ultimately show the codec units of the same shape divided from the first codec unit 1200. In this way, by recursively dividing the codec units in different ways based on the division shape pattern information, the image decoding device 100 can process a plurality of codec units in different orders even if the codec units are ultimately determined to be the same shape.

[0176] 13 illustrates a process of determining a depth of a codec unit when a shape and a size of a codec unit are changed when a codec unit is recursively divided so that a plurality of codec units are determined according to an embodiment.

[0177] According to an embodiment, the image decoding device 100 may determine the depth of the codec unit based on a preset criterion. For example, the preset criterion may be the length of the long side of the codec unit. When the length of the long side of the codec unit before being divided is 2n times (n>0) the length of the long side of the current codec unit divided, the image decoding device 100 may determine that the depth of the current codec unit is increased by n from the depth of the codec unit before being divided. In the following description, a codec unit with an increased depth is represented as a codec unit of a deeper depth.

[0178] 13, according to an embodiment, the image decoding device 100 may determine a second codec unit 1302 and a third codec unit 1304 of a deeper depth by dividing a square first codec unit 1300 based on block shape information indicating a square shape (for example, the block shape information may be represented as '0: SQUARE'). Assuming that the size of the square first codec unit 1300 is 2N×2N, the second codec unit 1302 determined by dividing by 1 / 2 of the width and height of the first codec unit 1300 may have a size of N×N. In addition, the third codec unit 1304 determined by dividing by 1 / 2 of the width and height of the second codec unit 1302 may have a size of N / 2×N / 2. In this case, the width and height of the third codec unit 1304 are 1 / 4 times the width and height of the first codec unit 1300. When the depth of the first codec unit 1300 is D, the depth of the second codec unit 1302 whose width and height are 1 / 2 times those of the first codec unit 1300 may be D+1, and the depth of the third codec unit 1304 whose width and height are 1 / 4 times those of the first codec unit 1300 may be D+2.

[0179] According to an embodiment, the image decoding device 100 can determine the second codec unit 1312 or 1322 and the third codec unit 1314 or 1324 of a deeper depth by dividing the non-square first codec unit 1310 or 1320 based on block shape information indicating a non-square shape (for example, the block shape information can be represented as '1: NS_VER' indicating a non-square shape, whose height is longer than the width, or the block shape information can be represented as '2: NS_HOR' indicating a non-square shape, whose width is longer than the height).

[0180] The image decoding device 100 may determine the second codec unit 1302, 1312, or 1322 by dividing at least one of the width and the height of the first codec unit 1310 having a size of N×2N. That is, the image decoding device 100 may determine the second codec unit 1302 having a size of N×N or the second codec unit 1322 having a size of N×N / 2 by dividing the first codec unit 1310 in the horizontal direction, or may determine the second codec unit 1312 having a size of N / 2×N by dividing the first codec unit 1310 in the horizontal direction and the vertical direction.

[0181] According to an embodiment, the image decoding device 100 may determine the second codec unit 1302, 1312, or 1322 by dividing at least one of the width and the height of the first codec unit 1320 having a size of 2N×N. That is, the image decoding device 100 may determine the second codec unit 1302 having a size of N×N or the second codec unit 1312 having a size of N / 2×N by dividing the first codec unit 1320 in the vertical direction, or may determine the second codec unit 1322 having a size of N×N / 2 by dividing the first codec unit 1320 in the horizontal direction and the vertical direction.

[0182] According to an embodiment, the image decoding device 100 may determine the third codec unit 1304, 1314, or 1324 by dividing at least one of the width and the height of the second codec unit 1302 having a size of N×N. That is, the image decoding device 100 may determine the third codec unit 1304 having a size of N / 2×N / 2, the third codec unit 1314 having a size of N / 4×N / 2, or the third codec unit 1324 having a size of N / 2×N / 4 by dividing the second codec unit 1302 in the vertical direction and the horizontal direction.

[0183] According to an embodiment, the image decoding device 100 may determine the third codec unit 1304, 1314, or 1324 by dividing at least one of the width and the height of the second codec unit 1312 having a size of N / 2×N. That is, the image decoding device 100 may determine the third codec unit 1304 having a size of N / 2×N / 2 or the third codec unit 1324 having a size of N / 2×N / 4 by dividing the second codec unit 1312 in the horizontal direction, or may determine the third codec unit 1314 having a size of N / 4×N / 2 by dividing the second codec unit 1312 in the vertical direction and the horizontal direction.

[0184] According to an embodiment, the image decoding device 100 may determine the third codec unit 1304, 1314, or 1324 by dividing at least one of the width and the height of the second codec unit 1322 having a size of N×N / 2. That is, the image decoding device 100 may determine the third codec unit 1304 having a size of N / 2×N / 2 or the third codec unit 1314 having a size of N / 4×N / 2 by dividing the second codec unit 1322 in the vertical direction, or may determine the third codec unit 1324 having a size of N / 2×N / 4 by dividing the second codec unit 1322 in the vertical direction and the horizontal direction.

[0185] According to an embodiment, the image decoding device 100 may divide the square codec unit 1300, 1302, or 1304 in the horizontal direction or the vertical direction. For example, the image decoding device 100 may determine the first codec unit 1310 of size N×2N by dividing the first codec unit 1300 of size 2N×2N in the vertical direction, or may determine the first codec unit 1320 of size 2N×N by dividing the first codec unit 1300 in the horizontal direction. According to an embodiment, when the depth is determined based on the length of the longest side of the codec unit, the depth of the codec unit determined by dividing the first codec unit 1300 of size 2N×2N in the horizontal direction or the vertical direction may be the same as the depth of the first codec unit 1300.

[0186] According to an embodiment, the width and height of the third codec unit 1314 or 1324 may be 1 / 4 times the width and height of the first codec unit 1310 or 1320. When the depth of the first codec unit 1310 or 1320 is D, the depth of the second codec unit 1312 or 1322 whose width and height are 1 / 2 times the first codec unit 1310 or 1320 may be D+1, and the depth of the third codec unit 1314 or 1324 whose width and height are 1 / 4 times the first codec unit 1310 or 1320 may be D+2.

[0187] FIG. 14 illustrates a depth that may be determined based on a shape and a size of a codec unit, and a partial index (PID) for distinguishing a codec unit according to an embodiment.

[0188] According to an embodiment, the image decoding device 100 may determine the second codec units of various shapes by dividing the square first codec unit 1400. Referring to FIG14, the image decoding device 100 may determine the second codec units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d by dividing the first codec unit 1400 in at least one of the vertical direction and the horizontal direction based on the division shape pattern information. That is, the image decoding device 100 may determine the second codec units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d based on the division shape pattern information of the first codec unit 1400.

[0189] According to an embodiment, the depth of the second codec units 1402a and 1402b, 1404a and 1404b, and 1406a, 1406b, 1406c, and 1406d determined based on the division shape pattern information of the square first codec unit 1400 can be determined based on the length of its long side. For example, because the side length of the square first codec unit 1400 is equal to the length of the long side of the non-square second codec units 1402a and 1402b and 1404a and 1404b, the first codec unit 1400 and the non-square second codec units 1402a and 1402b and 1404a and 1404b can have the same depth, for example, D. However, when the image decoding device 100 divides the first codec unit 1400 into four square second codec units 1406a, 1406b, 1406c and 1406d based on the division shape pattern information, because the side length of the square second codec units 1406a, 1406b, 1406c and 1406d is 1 / 2 times the side length of the first codec unit 1400, the depth of the second codec units 1406a, 1406b, 1406c and 1406d can be D+1, which is 1 deeper than the depth D of the first codec unit 1400.

[0190] According to an embodiment, the image decoding device 100 may determine a plurality of second codec units 1412a and 1412b and 1414a, 1414b and 1414c by dividing the first codec unit 1410 having a height greater than a width in a horizontal direction based on the division shape pattern information. According to an embodiment, the image decoding device 100 may determine a plurality of second codec units 1422a and 1422b and 1424a, 1424b and 1424c by dividing the first codec unit 1420 having a width greater than a height in a vertical direction based on the division shape pattern information.

[0191] According to an embodiment, the depth of the second codec units 1412a and 1412b and 1414a, 1414b and 1414c, or 1422a and 1422b and 1424a, 1424b and 1424c determined based on the division shape pattern information of the non-square first codec unit 1410 or 1420 can be determined based on the length of its long side. For example, because the side length of the square second codec units 1412a and 1412b is 1 / 2 times the length of the long side of the non-square first codec unit 1410 whose height is greater than the width, the depth of the square second codec units 1412a and 1412b is D+1, which is 1 deeper than the depth D of the non-square first codec unit 410.

[0192] In addition, the image decoding device 100 may divide the non-square first codec unit 1410 into an odd number of second codec units 1414a, 1414b, and 1414c based on the division shape pattern information. The odd number of second codec units 1414a, 1414b, and 1414c may include the non-square second codec units 1414a and 1414c and the square second codec unit 1414b. In this case, since the length of the long side of the non-square second codec units 1414a and 1414c and the length of the side of the square second codec unit 1414b are 1 / 2 times the length of the long side of the non-square first codec unit 1410, the depth of the second codec units 1414a, 1414b, and 1414c may be D+1, which is 1 deeper than the depth D of the non-square first codec unit 1410. The image decoding device 100 may determine the depth of the codec unit split from the first codec unit 1420 having a non-square shape having a width greater than a height by using the above-described method of determining the depth of the codec unit split from the first codec unit 1410 .

[0193] According to an embodiment, when the sizes of the odd-numbered divided codec units are not equal, the image decoding device 100 may determine the PID for identifying the divided codec units based on the size ratio between the codec units. Referring to FIG. 14, the codec unit 1414b at the center position among the odd-numbered divided codec units 1414a, 1414b, and 1414c may have the same width as the other codec units 1414a and 1414c and twice the height of the other codec units 1414a and 1414c. That is, in this case, the codec unit 1414b at the center position may include two of the other codec units 1414a or 1414c. Therefore, when the PID of the codec unit 1414b at the center position is 1 based on the scanning order, the PID of the codec unit 1414c located next to the codec unit 1414b may be increased by 2, and thus may be 3. That is, there may be discontinuity in the PID value. According to an embodiment, the image decoding apparatus 100 may determine whether odd-numbered divided codec units have equal sizes based on whether there is discontinuity in a PID for identifying the divided codec units.

[0194] According to an embodiment, the image decoding device 100 may determine whether to use a specific division method based on a PID value for identifying a plurality of codec units determined by dividing the current codec unit. Referring to FIG. 14 , the image decoding device 100 may determine an even number of codec units 1412a and 1412b or an odd number of codec units 1414a, 1414b, and 1414c by dividing a first codec unit 1410 having a rectangular shape having a height greater than a width. The image decoding device 100 may use a PID indicating a corresponding codec unit in order to identify the corresponding codec unit. According to an embodiment, the PID may be obtained from a sample point (e.g., an upper left sample point) at a preset position of each codec unit.

[0195] According to an embodiment, the image decoding device 100 may determine a codec unit at a preset position in the divided codec unit by using a PID for distinguishing codec units. According to an embodiment, when the division shape pattern information of the first codec unit 1410 having a rectangle with a height greater than a width indicates that the codec unit is divided into three codec units, the image decoding device 100 may divide the first codec unit 1410 into three codec units 1414a, 1414b, and 1414c. The image decoding device 100 may assign a PID to each of the three codec units 1414a, 1414b, and 1414c. The image decoding device 100 may compare the PIDs of the odd-numbered divided codec units to determine the codec unit at the center position in the codec unit. The image decoding device 100 may determine the codec unit 1414b having a PID corresponding to an intermediate value in the PID of the codec unit as the codec unit at the center position in the codec unit determined by dividing the first codec unit 1410. According to an embodiment, when the divided codec units do not have equal sizes, the image decoding device 100 may determine the PID for distinguishing the divided codec units based on the size ratio between the codec units. Referring to FIG. 14, the codec unit 1414b generated by dividing the first codec unit 1410 may have the same width as the other codec units 1414a and 1414c and twice the height of the other codec units 1414a and 1414c. In this case, when the PID of the codec unit 1414b at the center position is 1, the PID of the codec unit 1414c located next to the codec unit 1414b may increase by 2, and thus may be 3. As described above, when the PID does not increase uniformly, the image decoding device 100 may determine that the codec unit is divided into a plurality of codec units, and the plurality of codec units include codec units having sizes different from those of the other codec units. According to an embodiment, when the division shape mode information indicates that the codec unit is divided into an odd number of codec units, the image decoding device 100 may divide the current codec unit in such a way that the codec unit at a preset position in the odd number of codec units (e.g., the codec unit at the center position) has a size different from that of other codec units. In this case, the image decoding device 100 may determine the codec unit at the center position having different sizes by using the PID of the codec unit. However, the PID and the size or position of the codec unit at the preset position are not limited to the above examples, and various PIDs and various positions and sizes of the codec units may be used.

[0196] According to an embodiment, the image decoding apparatus 100 may use a preset data unit that is recursively divided starting from a coding unit.

[0197] FIG. 15 illustrates determining a plurality of coding units based on a plurality of preset data units included in a picture according to an embodiment.

[0198] According to an embodiment, a preset data unit may be defined as a data unit that recursively divides a codec unit by starting with division shape pattern information. That is, the preset data unit may correspond to a codec unit of the highest depth, which is used to determine a plurality of codec units divided from a current picture. In the following description, for ease of explanation, the preset data unit is referred to as a reference data unit.

[0199] According to an embodiment, the reference data unit may have a preset size and a preset shape. According to an embodiment, the reference codec unit may include M×N samples. Here, M and N may be equal to each other and may be an integer expressed as a power of 2. That is, the reference data unit may have a square or non-square shape and may be divided into an integer number of codec units.

[0200] According to an embodiment, the image decoding apparatus 100 may divide the current picture into a plurality of reference data units. According to an embodiment, the image decoding apparatus 100 may divide the plurality of reference data units divided from the current picture by using the division shape pattern information of each reference data unit. The operation of dividing the reference data unit may correspond to the division operation using the quadtree structure.

[0201] According to an embodiment, the image decoding device 100 may predetermine the minimum size allowed for the reference data unit included in the current picture. Therefore, the image decoding device 100 may determine various reference data units whose sizes are equal to or larger than the minimum size, and may determine one or more coding units by using the division shape mode information with respect to the determined reference data unit.

[0202] 15 , the image decoding apparatus 100 may use a square reference codec unit 1500 or a non-square reference codec unit 1502. According to an embodiment, the shape and size of the reference codec unit may be determined based on various data units that may include one or more reference codec units (e.g., a sequence, a picture, a slice, a slice segment, a slice, a slice group, a maximum codec unit, etc.).

[0203] According to an embodiment, the receiver 110 of the image decoding device 100 may obtain at least one of the reference codec unit shape information and the reference codec unit size information with respect to each of the various data units from the bitstream. With respect to the operation of dividing the current codec unit 300 of FIG. 3 , the operation of dividing the square reference codec unit 1500 into one or more codec units is described above, and with respect to the operation of dividing the current codec unit 400 or 450 of FIG. 4 , the operation of dividing the non-square reference codec unit 1502 into one or more codec units is described above. Therefore, a detailed description thereof will not be provided herein.

[0204] According to an embodiment, the image decoding device 100 may use a PID for identifying the size and shape of a reference codec unit to determine the size and shape of the reference codec unit according to some data units predetermined based on a preset condition. That is, the receiver 110 may obtain a PID only from a bitstream to identify the size and shape of a reference codec unit for each slice, slice segment, slice, slice group, or maximum codec unit as a data unit satisfying a preset condition (a data unit having a size equal to or less than a slice) among various data units (e.g., a sequence, a picture, a slice, a slice segment, a slice, a slice group, a sub-maximum codec unit, etc.). The image decoding device 100 may determine the size and shape of a reference data unit with respect to each data unit satisfying the preset condition by using a PID. When the reference codec unit shape information and the reference codec unit size information are obtained and used from a bitstream according to each data unit having a relatively small size, the efficiency of using the bitstream may not be high, and therefore, only the PID may be obtained and used instead of directly obtaining the reference codec unit shape information and the reference codec unit size information. In this case, at least one of the size and shape of the reference codec unit corresponding to the PID for identifying the size and shape of the reference codec unit may be predetermined. That is, the image decoding apparatus 100 may determine at least one of the size and shape of the reference codec unit included in the data unit serving as the unit for obtaining the PID by selecting at least one of the size and shape of the reference codec unit determined in advance based on the PID.

[0205] According to an embodiment, the image decoding device 100 may use one or more reference codec units included in the maximum codec unit. That is, the maximum codec unit divided from the picture may include one or more reference codec units, and the codec unit may be determined by recursively dividing each reference codec unit. According to an embodiment, at least one of the width and height of the maximum codec unit may be an integer multiple of at least one of the width and height of the reference codec unit. According to an embodiment, the size of the reference codec unit may be obtained by dividing the maximum codec unit n times based on a quadtree structure. That is, according to various embodiments, the image decoding device 100 may determine the reference codec unit by dividing the maximum codec unit n times based on a quadtree structure, and may divide the reference codec unit based on at least one of block shape information and division shape pattern information.

[0206] FIG. 16 illustrates a processing block used as a criterion for determining an order of determining reference codec units included in a picture according to an embodiment.

[0207] According to an embodiment, the image decoding device 100 may determine one or more processing blocks divided from a picture. A processing block is a data unit including one or more reference codec units divided from a picture, and one or more reference codec units included in the processing block may be determined according to a specific order. That is, the determination order of the one or more reference codec units determined in each processing block may correspond to one of the various types of orders for determining the reference codec units, and may vary depending on the processing block. The determination order of the reference codec units determined relative to each processing block may be one of various orders (for example, a raster scan order, a Z scan, an N scan, an upper right diagonal scan, a horizontal scan, and a vertical scan, but not limited to the scan order mentioned).

[0208] According to an embodiment, the image decoding device 100 may obtain processing block size information and may determine the size of one or more processing blocks included in a picture. The image decoding device 100 may obtain processing block size information from a bitstream and may determine the size of one or more processing blocks included in a picture. The size of the processing block may be a preset size of a data unit indicated by the processing block size information.

[0209] According to an embodiment, the receiver 110 of the image decoding device 100 may obtain processing block size information from a bitstream according to each specific data unit. For example, the processing block size information may be obtained from a bitstream in a data unit such as an image, a sequence, a picture, a slice, a slice segment, a slice, or a slice group. That is, the receiver 110 may obtain processing block size information from a bitstream according to each of the various data units, and the image decoding device 100 may determine the size of one or more processing blocks divided from the picture by using the obtained processing block size information. The size of the processing block may be an integer multiple of the reference codec unit.

[0210] According to an embodiment, the image decoding device 100 may determine the size of the processing blocks 1602 and 1612 included in the picture 1600. For example, the image decoding device 100 may determine the size of the processing block based on the processing block size information obtained from the bitstream. Referring to FIG. 16, according to an embodiment, the image decoding device 100 may determine the width of the processing blocks 1602 and 1612 to be four times the width of the reference codec unit, and may determine the height of the processing blocks 1602 and 1612 to be four times the height of the reference codec unit. The image decoding device 100 may determine the determination order of one or more reference codec units in one or more processing blocks.

[0211] According to an embodiment, the image decoding device 100 may determine the processing blocks 1602 and 1612 included in the picture 1600 based on the size of the processing block, and may determine the determination order of one or more reference codec units in the processing blocks 1602 and 1612. According to an embodiment, the determination of the reference codec unit may include determination of the size of the reference codec unit.

[0212] According to an embodiment, the image decoding device 100 may obtain the determination order information of one or more reference codec units included in one or more processing blocks from the bitstream, and may determine the determination order of the one or more reference codec units based on the obtained determination order information. The determination order information may be defined as an order or direction for determining the reference codec units in the processing block. That is, the determination order of the reference codec units may be determined independently with respect to each processing block.

[0213] According to an embodiment, the image decoding apparatus 100 may obtain the determination order information of the reference codec unit according to each specific data unit from the bitstream. For example, the receiver 110 may obtain the determination order information of the reference codec unit from the bitstream according to each data unit (such as an image, a sequence, a picture, a slice, a slice segment, a slice, a slice group, or a processing block). Since the determination order information of the reference codec unit indicates the order for determining the reference codec unit in the processing block, the determination order information may be obtained with respect to each specific data unit including an integer number of processing blocks.

[0214] According to an embodiment, the image decoding apparatus 100 may determine one or more reference coding units based on the determined determination order.

[0215] According to an embodiment, the receiver 110 may obtain the determination order information of the reference codec unit from the bitstream as information related to the processing blocks 1602 and 1612, and the image decoding device 100 may determine the determination order of one or more reference codec units included in the processing blocks 1602 and 1612, and determine the one or more reference codec units included in the picture 1600 based on the determination order. Referring to FIG. 16, the image decoding device 100 may determine the determination order 1604 and 1614 of one or more reference codec units in the processing blocks 1602 and 1612, respectively. For example, when the determination order information of the reference codec unit is obtained with respect to each processing block, the determination order information of different types of reference codec units may be obtained with respect to the processing blocks 1602 and 1612. When the determination order 1604 of the reference codec units in the processing block 1602 is a raster scan order, the reference codec units included in the processing block 1602 may be determined according to the raster scan order. In contrast, when the determination order 1614 of the reference codec units in another processing block 1612 is a backward raster scan order, the reference codec units included in the processing block 1612 may be determined according to the backward raster scan order.

[0216] According to an embodiment, the image decoding apparatus 100 may decode the determined one or more reference codec units. As described above, the image decoding apparatus 100 may decode the image based on the determined reference codec unit. The method of decoding the reference codec unit may include various image decoding methods.

[0217] According to an embodiment, the image decoding device 100 may obtain block shape information indicating the shape of the current codec unit or division shape pattern information indicating the division method of the current codec unit from the bitstream, and may use the obtained information. The division shape pattern information may be included in the bitstream associated with various data units. For example, the image decoding device 100 may use the division shape pattern information included in a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a slice header, or a slice group header. In addition, the image decoding device 100 may obtain a syntax element corresponding to the block shape information or the division shape pattern information from the bitstream according to each maximum codec unit, each reference codec unit, or each processing block, and may use the obtained syntax element.

[0218] Next, a method for determining a division rule according to an embodiment of the present invention will be described in detail.

[0219] The image decoding device 100 may determine a division rule of the image. The division rule may be predetermined between the image decoding device 100 and the image encoding device 2200. The image decoding device 100 may determine the division rule of the image based on information obtained from the bitstream. The image decoding device 100 may determine the division rule based on information obtained from at least one of a sequence parameter set, a picture parameter set, a video parameter set, a slice header, a slice segment header, a slice header, and a slice group header. The image decoding device 100 may determine the division rule differently according to a frame, a slice, a slice, a temporal layer, a maximum codec unit, or a codec unit.

[0220] The image decoding device 100 may determine the division rule based on the block shape of the codec unit. The block shape may include the size, shape, aspect ratio, and direction of the codec unit. The image encoding device 2200 and the image decoding device 100 may determine the division rule based on the block shape of the codec unit in advance. However, the embodiment is not limited to this. The image decoding device 100 may determine the division rule based on information obtained from the bit stream received from the image encoding device 2200.

[0221] The shape of the codec unit may include a square and a non-square. When the width and height of the codec unit are the same, the image decoding device 100 may determine the shape of the codec unit as a square. In addition, when the width and height of the codec unit are different, the image decoding device 100 may determine the shape of the codec unit as a non-square.

[0222] The size of the codec unit may include various sizes, such as 4x4, 8x4, 4x8, 8x8, 16x4, 16x8, and 256x256. The size of the codec unit may be classified based on the length of the long side, the length of the short side, or the area of ​​the codec unit. The image decoding device 100 may apply the same division rule to the codec units classified into the same group. For example, the image decoding device 100 may classify codec units having long sides of the same length as codec units having the same size. In addition, the image decoding device 100 may apply the same division rule to codec units having long sides of the same length.

[0223] The aspect ratio of the codec unit may include 1:2, 2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 32:1, 1:32, etc. In addition, the direction of the codec unit may include a horizontal direction and a vertical direction. The horizontal direction may indicate that the length of the width of the codec unit is greater than the length of its height. The vertical direction may indicate that the length of the width of the codec unit is less than the length of its height.

[0224] The image decoding device 100 may adaptively determine the division rule based on the size of the codec unit. The image decoding device 100 may determine the allowed division shape mode differently based on the size of the codec unit. For example, the image decoding device 100 may determine whether to allow division based on the size of the codec unit. The image decoding device 100 may determine the division direction according to the size of the codec unit. The image decoding device 100 may determine the allowed division type according to the size of the codec unit.

[0225] The division rule determined based on the size of the coding unit may be a division rule predetermined between the image encoding device 2200 and the image decoding device 100. In addition, the image decoding device 100 may determine the division rule based on information obtained from a bitstream.

[0226] The image decoding apparatus 100 may adaptively determine the division rule based on the position of the coding unit. The image decoding apparatus 100 may adaptively determine the division rule based on the position of the coding unit in the image.

[0227] In addition, the image decoding device 100 may determine a division rule so that the codec units generated via different division paths do not have the same block shape. However, the embodiment is not limited thereto, and the codec units generated via different division paths have the same block shape. The codec units generated via different division paths may have different decoding processing orders. Since the decoding processing order is described above with reference to FIG. 12, its details are not provided.

[0228] Hereinafter, according to an embodiment of the present specification, a method for dividing a codec unit for efficient hardware implementation is provided, and a method and apparatus for encoding or decoding a video will be described in detail with reference to FIGS. 17 to 20 .

[0229] FIG. 17 shows a block diagram of a video encoding apparatus according to an embodiment.

[0230] According to an embodiment, the video encoding device 1700 may include a memory 1710 and at least one processor 1720 connected to the memory 1710. The operation of the video encoding device 1700 according to the embodiment may be performed as a separate processor or may be performed under the control of a central processor. In addition, the memory 1710 of the video encoding device 1700 may store data received from the outside and data generated by the processor, for example, partition information related to the codec unit, prediction information related to the codec unit, and residual information related to the codec unit, information on whether binary partitioning of the codec unit is allowed, information on whether ternary partitioning of the codec unit is allowed, etc.

[0231] In order to efficiently encode an image divided into a plurality of blocks, the video encoding device 1700 may perform encoding using a pipeline method of performing processing of a plurality of blocks in parallel. Hereinafter, a block processed during one pipeline cycle to implement pipelined video encoding in hardware is referred to as a pipeline data unit. The size of the pipeline data unit may be predetermined.

[0232] Because the processor 1720 can access the data of the current pipeline data unit during the current pipeline cycle, the block located in the pipeline can be encoded. However, because the processor 1720 may not be able to access the data of the next pipeline data unit or the previous pipeline data unit during the current pipeline cycle, the block spanning the current pipeline data unit and the next pipeline data unit, or the block spanning the current pipeline data unit and the previous pipeline data unit may not be encoded.

[0233] Furthermore, in order to be able to perform encoding on a block in a pipeline data unit within a pipeline cycle, the size of a codec unit or a transform unit needs to be smaller than or equal to the size of the pipeline data unit.

[0234] In addition, when the origin of the upper left position of the codec unit includes the outline of the image, the codec unit may be outside the image depending on the size of the codec unit, and thus the video encoding device 1700 may determine to arbitrarily divide the codec unit.

[0235] The processor 1720 of the video encoding device 1700 can determine the codec unit by dividing at least one of the height and width of the maximum codec unit having the first size based on whether the height or width of a non-square first codec unit including the outer boundary of the image in the codec unit is greater than the maximum transformation size, determine whether it is allowed to generate two second codec units by dividing at least one of the height and width of the first codec unit, and perform encoding by transforming the second codec unit generated from the first codec unit using the maximum transformation size.

[0236] In this regard, the maximum transform size may be determined to be less than or equal to the size of the pipeline data unit in consideration of the size of the pipeline data unit. Therefore, encoding of each second codec unit having a size less than or equal to the maximum transform size may be processed during one pipeline cycle, and therefore, the video encoding apparatus 1700 may process encoding using a pipeline method.

[0237] Hereinafter, a detailed operation of a method in which the video encoding apparatus 1700 performs encoding using a codec unit determined from a maximum codec unit will be described in detail with reference to FIG. 18 .

[0238] FIG. 18 shows a flowchart of a video encoding method according to an embodiment.

[0239] In operation 1810 , the video encoding device 1700 may generate a codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size.

[0240] In operation 1820, based on whether the height or width of the non-square first codec unit including the outer boundary of the image in the codec unit is greater than the maximum transform size, the video encoding device 1700 may determine whether it is allowed to generate two second codec units by dividing at least one of the height and width of the first codec unit.

[0241] According to an embodiment, the video encoding apparatus 1700 may set a flag value indicating whether it is allowed to generate the second codec unit by dividing at least one of the height and the width of the non-square first codec unit including the outer boundary of the image.

[0242] In operation 1820, according to an embodiment, when the length of the width of the non-square first codec unit including the outer boundary of the image is greater than the maximum transformation size and the length of the height of the first codec unit is equal to the maximum transformation size, the video encoding device 1700 may allow a smaller codec unit to be generated by performing vertical binary division on the first codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the first codec unit. Specifically, the video encoding device 1700 may set a vertical binary division flag value for allowing the generation of a smaller codec unit by performing vertical binary division on the first codec unit, and set a horizontal binary division flag value for prohibiting the generation of a smaller codec unit by performing horizontal binary division on the first codec unit.

[0243] According to an embodiment, when the length of the width of the non-square first codec unit including the outer boundary of the image is equal to the maximum transformation size and the length of the height of the first codec unit is greater than the maximum transformation size, the video encoding device 1700 may allow a smaller codec unit to be generated by performing horizontal binary division on the first codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the first codec unit. Specifically, the video encoding device 1700 may set a horizontal binary division flag value to allow the generation of a smaller codec unit by performing horizontal binary division on the first codec unit, and set a vertical binary division flag value to prohibit the generation of a smaller codec unit by performing vertical binary division on the first codec unit.

[0244] According to an embodiment, when the length of the height of the codec unit including the outer boundary of the image is greater than the maximum transformation size and the length of the width of the codec unit is greater than the maximum transformation size, the video encoding device 1700 may allow a smaller codec unit to be generated by performing quadtree partitioning on the codec unit.

[0245] According to an embodiment, the size of the height and width of the coding unit may be greater than or equal to the maximum transform size.

[0246] In addition, in operation 1820, when the first codec unit is adjacent to the outline of the image so that the height or width of the first codec unit is outside the outline of the image, the video encoding device 1700 may prohibit a binary division in a direction of dividing the height or width of the first codec unit, which may be outside the outline of the image. As a specific example, when the height of the first codec unit is greater than the maximum transform size and the width of the first codec unit is outside the right outline of the image, the video encoding device 1700 may determine to prohibit a vertical binary division for dividing the width of the first codec unit. Similarly, when the width of the first codec unit is greater than the maximum transform size and the height of the first codec unit is outside the lower outline of the image, the video encoding device 1700 may determine to prohibit a horizontal binary division for dividing the height of the first codec unit.

[0247] However, when the height of the codec unit including the outer boundary of the image is outside the lower contour of the image or the width of the codec unit is outside the right contour of the image, the video encoding device 1700 may allow quadtree partitioning of the codec unit.

[0248] In operation 1830, the video encoding device 1700 may perform encoding by transforming the second codec unit generated from the first codec unit using the maximum transform size. The video encoding device 1700 may determine a prediction block by performing prediction on the second codec unit, and determine a residual block composed of a difference between a sample value of an original block of the second codec unit and a sample value of a prediction block. The video encoding device 1700 may generate a quantized transform coefficient by performing transformation and quantization on the residual block, and encode the second codec unit by performing entropy encoding on the quantized transform coefficient.

[0249] According to an embodiment, the video encoding device 1700 may also determine whether to arbitrarily divide the codec unit into codec units of lower depths, taking into account the size of the pipeline data unit. In order to complete the transformation and quantization of the residual data in the codec unit during one pipeline cycle, it is necessary to determine that the size of the pipeline data unit is equal to the size of the maximum transformation unit. In addition, when the pipeline data unit and the maximum transformation unit have a square shape, the height and width of the pipeline data unit and the maximum transformation unit are equal to each other. Therefore, when the length of the height or width of the codec unit is greater than the length of the height or width of the maximum transformation unit, by arbitrarily dividing the height or width of the codec unit to divide the codec unit into codec units of lower depth, the encoding process of the corresponding codec unit can be completed within one pipeline cycle.

[0250] Therefore, an embodiment will be described below, in which, in order to complete the transformation within the pipeline cycle, the video encoding device 1700 compares the maximum transformation size with at least one of the height and width of the codec unit, and sets the vertical binary division flag value, the horizontal binary division flag value and the quadtree division flag value to respectively indicate whether vertical binary division, horizontal binary division and quadtree division of the codec unit are allowed.

[0251] Specifically, in order to complete the transformation within the pipeline cycle, when the length of the width of the codec unit is greater than the maximum transformation size and the length of the height of the codec unit is equal to the maximum transformation size, the video encoding device 1700 may allow the generation of a smaller codec unit by performing vertical binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the codec unit.

[0252] Specifically, in order to complete the transformation within the pipeline cycle, when the length of the width of the codec unit is equal to the maximum transformation size and the length of the height of the codec unit is greater than the maximum transformation size, the video encoding device 1700 may allow the generation of a smaller codec unit by performing horizontal binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the codec unit.

[0253] Specifically, in order to complete the transformation within the pipeline cycle, when the length of the width of the codec unit is greater than the maximum transformation size and the length of the height of the codec unit is equal to the maximum transformation size, the video encoding device 1700 may allow the generation of a smaller codec unit by performing vertical binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the codec unit.

[0254] Specifically, in order to complete the transformation within the pipeline cycle, when the length of the width of the codec unit is equal to the maximum transformation size and the length of the height of the codec unit is greater than the maximum transformation size, the video encoding device 1700 may allow the generation of a smaller codec unit by performing horizontal binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the codec unit.

[0255] Specifically, in order to complete the transformation within the pipeline cycle, when each of the dimensions of the height and width of the codec unit is larger than the maximum transformation size, the video encoding device 1700 may allow a smaller codec unit to be generated by performing quadtree partitioning on the codec unit.

[0256] In addition, according to an embodiment, the video encoding device 1700 may predetermine a maximum value and a minimum value to limit the size of a maximum codec unit or a codec unit, and encode information about the maximum value and the minimum value.

[0257] In operation 1810, according to an embodiment, the video encoding device 1700 may divide an image into maximum codec units having a constant size and encode the image. Each maximum codec unit may be divided into codec units having an optimal size, from which the highest codec efficiency may be obtained. To this end, the size of the maximum codec unit may be predetermined, and the video encoding device 1700 may encode information about the size of the maximum codec unit and include the encoded information in a sequence parameter set (SPS). According to an embodiment, the size of the maximum codec unit may be 128x128.

[0258] According to an embodiment, the video encoding device 1700 may differently determine a maximum size and a minimum size of a coding unit based on a type of a slice.

[0259] Specifically, the video encoding device 1700 may set the minimum size of the luma codec unit generated by performing binary division on the intra slice and the maximum size of the luma codec unit. In addition, by using the minimum size of the luma codec unit generated by performing binary division on the intra slice and the maximum size of the luma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra slice. In addition, the video encoding device 1700 may encode information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra slice, and include the encoded information in the SPS.

[0260] Specifically, the video encoding device 1700 may pre-set the maximum size of the luma codec unit generated by performing three-way division on the intra slice. In addition, by using the minimum size of the luma codec unit generated by performing three-way division on the intra slice and the maximum size of the luma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the intra slice. In addition, the video encoding device 1700 may encode information about the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the intra slice, and include the encoded information in the SPS.

[0261] Specifically, the video encoding device 1700 may pre-set the maximum size of the luma codec unit generated by performing binary division on the inter-frame slice. In addition, by using the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice and the maximum size of the luma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice. In addition, the video encoding device 1700 may encode information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice, and include the encoded information in the SPS.

[0262] Specifically, the video encoding device 1700 may pre-set the maximum size of the luma codec unit generated by performing three-way division on the inter-frame slice. In addition, by using the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice and the maximum size of the luma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice. In addition, the video encoding device 1700 may encode information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice, and include the encoded information in the SPS.

[0263] According to an embodiment, the video encoding device 1700 may determine whether the luma codec unit and the chroma codec unit are determined separately. The video encoding device 1700 may encode information indicating whether the luma codec unit and the chroma codec unit are determined separately, and include the encoded information in the SPS.

[0264] According to an embodiment, when the luma codec unit and the chroma codec unit are determined separately, the video encoding device 1700 may pre-set the maximum size of the chroma codec unit generated by performing binary division on the intra slice. In addition, by using the minimum size of the chroma codec unit generated by performing binary division on the intra slice and the maximum size of the chroma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing binary division on the intra slice. In addition, the video encoding device 1700 may encode information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing binary division on the intra slice, and include the encoded information in the SPS.

[0265] According to an embodiment, when the luma codec unit and the chroma codec unit are determined separately, the video encoding device 1700 may pre-set the maximum size of the chroma codec unit generated by performing three-way division, and the three-way division may be determined in the intra slice. By using the minimum size of the chroma codec unit generated by performing three-way division on the intra slice and the maximum size of the chroma codec unit, the video encoding device 1700 may determine information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing three-way division on the intra slice. In addition, the video encoding device 1700 may encode information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing three-way division on the intra slice, and include the encoded information in the SPS.

[0266] According to an embodiment, when a first codec unit among the codec units is not adjacent to an outer boundary of the image, the video encoding device 1700 may encode the partition type information of the first codec unit and output the encoded partition type information in a bitstream.

[0267] Specifically, when binary partitioning is performed on the first codec unit, the video encoding device 1700 may determine two second codec units, each of which has a depth that is increased by 1 compared to the depth of the first codec unit, and encode the partition type information of the first codec unit indicating the binary partitioning. For example, when the depth of the first codec unit is N, the depth of each of the two second codec units generated by performing binary partitioning on the first codec unit may be N+1. In this regard, the area of ​​each of the second codec units may be 1 / 2 of the area of ​​the first codec unit.

[0268] Specifically, when a three-way division is performed on a first codec unit, the video encoding device 1700 may determine a third codec unit whose depth is increased by 1 compared to the depth of the first codec unit and two fourth codec units whose depth is increased by 2 compared to the depth of the first codec unit, and encode the division type information of the first codec unit indicating the three-way division. For example, when the depth of the first codec unit is N, the depth of the third codec unit may be N+1, and the depth of each of the two fourth codec units may be N+2, and the third codec unit and the fourth codec unit are generated by performing a three-way division on the first codec unit. In this regard, the area of ​​the third codec unit may be 1 / 2 of the area of ​​the first codec unit, and the area of ​​each of the fourth codec units may be 1 / 4 of the area of ​​the first codec unit.

[0269] Next, the video decoding process will be described in detail with reference to FIGS. 19 and 20 .

[0270] FIG. 19 shows a block diagram of a video decoding apparatus according to an embodiment.

[0271] According to an embodiment, the video decoding device 1900 may include a memory 1910 and at least one processor 1920 connected to the memory 1910. The operation of the video decoding device 1900 according to the embodiment may be performed as a separate processor or may be performed under the control of a central processor. In addition, the memory 1910 of the video decoding device 1900 may store data received from the outside and data generated by the processor, for example, partition information, prediction information, and residual information related to the codec unit, information on whether binary partitioning of the codec unit is allowed, information on whether ternary partitioning of the codec unit is allowed, etc.

[0272] In order to efficiently perform video decoding, the video decoding apparatus 1900 may perform decoding in a pipeline method of performing processing of a plurality of blocks in parallel, similar to the video encoding apparatus 1700. To this end, a pipeline data unit of the same size as that determined by the video encoding apparatus 1700 may be used.

[0273] The processor 1920 can access the data of the current pipeline data unit during the current pipeline cycle, and thus can decode the block located in the pipeline. However, the processor 1920 may not be able to decode the block across the current pipeline data unit and the next pipeline data unit, or to decode the block across the current pipeline data unit and the previous pipeline data unit.

[0274] In addition, as in the video encoding apparatus 1700 , the size of the coding unit or the transform unit needs to be smaller than or equal to the size of the pipeline data unit.

[0275] In addition, the video decoding apparatus 1900 may determine a division method for dividing a codec unit into codec units of a lower depth according to division type information obtained from a bitstream.

[0276] However, when the origin of the upper left position of the codec unit includes the outline of the image, the codec unit may be outside the image depending on the size of the codec unit, so the video decoding device 1900 may determine to arbitrarily divide the codec unit into a shape in which the codec unit does not cross the outline of the image, even in the absence of the division type information. Even if the division type information of the codec unit exists, the video decoding device 1900 may ignore the value of the division type information and divide the codec unit into a division shape in which the codec unit does not cross the outline of the image. When the video decoding device 1900 determines to arbitrarily divide the codec unit, the value of the division type information may be changed and set to an arbitrarily determined division shape.

[0277] The processor 1920 of the video decoding device 1900 can determine the codec unit by dividing at least one of the height and width of the maximum codec unit having the first size based on whether the height or width of a non-square first codec unit including the outer boundary of the image in the codec unit is greater than the maximum transformation size, determine whether it is allowed to generate two second codec units by dividing at least one of the height and width of the first codec unit, and perform decoding by inverse transforming the second codec unit generated from the first codec unit using the maximum transformation size.

[0278] In this regard, the maximum transform size may be determined to be less than or equal to the size of the pipeline data unit in consideration of the size of the pipeline data unit. Therefore, decoding of each second codec unit having a size less than or equal to the maximum transform size may be processed during one pipeline cycle, and therefore, the video decoding apparatus 1900 may process decoding using a pipeline method.

[0279] Hereinafter, detailed operations of a method in which the video decoding apparatus 1900 performs decoding using a codec unit determined from a maximum codec unit will be described in detail with reference to FIG. 20 .

[0280] FIG. 20 shows a flowchart of a video decoding method according to an embodiment.

[0281] In operation 2010, the video decoding device 1900 according to an embodiment may generate a codec unit by dividing at least one of a height and a width of a maximum codec unit having a first size.

[0282] In operation 2020, based on whether the height or width of the non-square first codec unit including the outer boundary of the image in the codec unit is greater than the maximum transformation size, the video decoding device 1900 can determine whether it is allowed to generate two second codec units by dividing at least one of the height and width of the first codec unit.

[0283] According to an embodiment, the video decoding device 1900 may set a flag value indicating whether it is allowed to generate the second codec unit by dividing at least one of the height and width of the first codec unit. Specifically, when the first codec unit includes the outer boundary of the image, depending on the size of the first codec unit, the first codec unit may be outside the outer boundary of the image, and therefore, the video decoding device 1900 may determine whether it is allowed to generate the second codec unit by arbitrarily dividing at least one of the height and width of the first codec unit.

[0284] According to an embodiment, when the length of the width of the non-square first codec unit including the outer boundary of the image is greater than the maximum transform size and the length of the height of the first codec unit is equal to the maximum transform size, the video decoding device 1900 may allow a smaller codec unit to be generated by performing vertical binary division on the first codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the first codec unit. Specifically, the video decoding device 1900 may set a vertical binary division flag value for allowing the generation of a smaller codec unit by performing vertical binary division on the first codec unit, and set a horizontal binary division flag value for prohibiting the generation of a smaller codec unit by performing horizontal binary division on the first codec unit.

[0285] According to an embodiment, when the length of the width of the non-square first codec unit including the outline of the image is equal to the maximum transformation size and the length of the height of the first codec unit is greater than the maximum transformation size, the video decoding device 1900 may allow a smaller codec unit to be generated by performing horizontal binary division on the first codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the first codec unit. Specifically, the video decoding device 1900 may set a horizontal binary division flag value for allowing the generation of a smaller codec unit by performing horizontal binary division on the first codec unit, and set a vertical binary division flag value for prohibiting the generation of a smaller codec unit by performing vertical binary division on the first codec unit.

[0286] According to an embodiment, when the length of each of the height and width of the codec unit including the outline of the image is greater than the maximum transform size, the video decoding device 1900 may allow a smaller codec unit to be generated by performing quadtree partitioning on the codec unit. Specifically, a quadtree partitioning flag value for allowing a smaller codec unit to be generated by performing quadtree partitioning on the codec unit may be set.

[0287] According to an embodiment, the length of the height and width of the coding unit may be greater than or equal to the maximum transform size.

[0288] In addition, in operation 2020, when the first codec unit is adjacent to the outline of the image so that the height or width of the first codec unit is outside the outline of the image, the video decoding device 1900 may prohibit a binary division in a direction that divides the height or width of the first codec unit, which may be outside the outline of the image. As a specific example, when the height of the first codec unit is greater than the maximum transform size and the width of the first codec unit is outside the right outline of the image, the video decoding device 1900 may determine to prohibit a vertical binary division for dividing the width of the first codec unit. Similarly, when the width of the first codec unit is greater than the maximum transform size and the height of the first codec unit is outside the lower outline of the image, the video decoding device 1900 may determine to prohibit a horizontal binary division for dividing the height of the first codec unit.

[0289] However, when the height of the codec unit including the outer boundary of the image is outside the lower contour of the image or the width of the codec unit is outside the right contour of the image, the video decoding device 1900 may allow quadtree partitioning of the codec unit.

[0290] In operation 2030, the video decoding device 1900 may perform decoding by inversely transforming the second codec unit generated from the first codec unit using the maximum transform size. The video decoding device 1900 may determine a prediction block by performing prediction on the second codec unit, reconstruct a residual block by performing inverse quantization and inverse transform on the second codec unit using the transform block, and determine a reconstructed block by synthesizing the prediction block and the residual block.

[0291] According to an embodiment, in response to a video encoding device 1700 in which the size of a pipeline data unit is set to be equal to a maximum transform size to complete encoding within a pipeline cycle, in order to enable the video decoding device 1900 to complete decoding within one pipeline cycle by performing an inverse transform and generating a reconstructed block, it is also necessary to set the size of the pipeline data unit used by the video decoding device 1900 to be equal to the maximum transform size, and to arbitrarily divide the encoding and decoding units based on the maximum transform size.

[0292] Therefore, an embodiment will be described below, in which, considering the pipeline cycle, the video decoding device 1900 compares the maximum transform size with at least one of the height and width of the codec unit, and sets a vertical binary division flag value, a horizontal binary division flag value, and a quadtree division flag value to respectively indicate whether vertical binary division, horizontal binary division, and quadtree division of the codec unit are allowed.

[0293] Specifically, in order to complete the decoding of the codec unit within the pipeline cycle, when the length of the width of the codec unit is greater than the maximum transform size and the length of the height of the codec unit is equal to the maximum transform size, the video decoding device 1900 may allow the generation of a smaller codec unit by performing vertical binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the codec unit.

[0294] Specifically, in order to complete the decoding of the codec unit within the pipeline cycle, when the length of the width of the codec unit is equal to the maximum transform size and the length of the height of the codec unit is greater than the maximum transform size, the video decoding device 1900 may allow the generation of a smaller codec unit by performing horizontal binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the codec unit.

[0295] Specifically, in order to complete the decoding of the codec unit within the pipeline cycle, when the length of the width of the codec unit is greater than the maximum transform size and the length of the height of the codec unit is equal to the maximum transform size, the video decoding device 1900 may allow the generation of a smaller codec unit by performing vertical binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing horizontal binary division on the codec unit.

[0296] Specifically, in order to complete the decoding of the codec unit within the pipeline cycle, when the length of the width of the codec unit is equal to the maximum transform size and the length of the height of the codec unit is greater than the maximum transform size, the video decoding device 1900 may allow the generation of a smaller codec unit by performing horizontal binary division on the codec unit, and prohibit the generation of a smaller codec unit by performing vertical binary division on the codec unit.

[0297] Specifically, in order to complete decoding of the codec unit within the pipeline cycle, when each of the height and width dimensions of the codec unit is greater than the maximum transform size, the video decoding device 1900 may allow a smaller codec unit to be generated by performing quadtree partitioning on the codec unit.

[0298] In addition, according to an embodiment, the video decoding device 1900 may predetermine a maximum value and a minimum value based on information obtained from a bitstream to limit the size of a maximum coding unit or a coding unit.

[0299] In operation 2010, according to an embodiment, the video decoding device 1900 may divide an image into a maximum codec unit having a constant size and decode the image. Since the size of the maximum codec unit has been previously determined by the video encoding device 1700, the video decoding device 1900 may obtain information about the size of the maximum codec unit from the SPS. According to an embodiment, the size of the maximum codec unit may be 128x128.

[0300] According to an embodiment, the video decoding device 1900 may separately obtain information about the maximum size and the minimum size of the coding unit from the bitstream based on the type of slice.

[0301] Specifically, the video decoding device 1900 can obtain information about the minimum size of the luma codec unit and information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra slice from the bitstream. By using the information about the minimum size of the luma codec unit and the information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra slice, the video decoding device 1900 can determine the maximum size of the luma codec unit generated by performing binary division on the intra slice. In addition, the video decoding device 1900 can obtain the encoded information about the minimum size of the luma codec unit and information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the intra slice from the SPS.

[0302] Specifically, the video decoding apparatus 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the intra slice from the bitstream. By using the information about the minimum size of the luma codec unit and the information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the intra slice, the video decoding apparatus 1900 may determine the maximum size of the luma codec unit generated by performing three-way division on the intra slice. In addition, the video decoding apparatus 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the intra slice from the SPS.

[0303] Specifically, the video decoding device 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice from the bitstream. By using the information about the minimum size of the luma codec unit and the information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice, the video decoding device 1900 may determine the maximum size of the luma codec unit generated by performing binary division on the inter-frame slice. In addition, the video decoding device 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing binary division on the inter-frame slice from the SPS.

[0304] Specifically, the video decoding device 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice from the bitstream. By using the information about the minimum size of the luma codec unit and the information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice, the video decoding device 1900 may determine the maximum size of the luma codec unit generated by performing three-way division on the inter-frame slice. In addition, the video decoding device 1900 may obtain information about the difference between the maximum size and the minimum size of the luma codec unit generated by performing three-way division on the inter-frame slice from the SPS.

[0305] According to an embodiment, the video decoding device 1900 may determine whether the luma codec unit and the chroma codec unit are determined separately based on information obtained from the bitstream. Specifically, the video decoding device 1900 may obtain information indicating whether the luma codec unit and the chroma codec unit are determined separately from the SPS.

[0306] Specifically, when the luma codec unit and the chroma codec unit are determined separately, the video decoding device 1900 can obtain information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing binary division on the intra slice from the bitstream. By using the information about the minimum size of the chroma codec unit and the information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing binary division on the intra slice, the video decoding device 1900 can determine the maximum size of the chroma codec unit generated by performing binary division on the intra slice. In addition, the video decoding device 1900 can obtain the encoded information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing binary division on the intra slice from the SPS.

[0307] Specifically, when the luma codec unit and the chroma codec unit are determined separately, the video decoding device 1900 can obtain information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing three-way division on the intra slice from the bitstream. By using the information about the minimum size of the chroma codec unit and the information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing three-way division on the intra slice, the video decoding device 1900 can determine the maximum size of the chroma codec unit generated by performing three-way division on the intra slice. In addition, the video decoding device 1900 can obtain information about the difference between the maximum size and the minimum size of the chroma codec unit generated by performing three-way division on the intra slice from the SPS.

[0308] According to an embodiment, when the coding unit is not adjacent to the outer boundary of the image, the video decoding device 1900 may obtain the encoding partition type information of the coding unit from the bitstream.

[0309] Specifically, when the obtained partition type information of the first codec unit indicates binary partition, the video decoding device 1900 may determine two second codec units whose depth is increased by 1 compared to the depth of the first codec unit. For example, when the depth of the first codec unit is N, the depth of each of the two second codec units generated by performing binary partition on the first codec unit may be N + 1. In this regard, the area of ​​each of the second codec units may be 1 / 2 of the area of ​​the first codec unit.

[0310] Specifically, when the partition type information of the obtained first codec unit indicates a trifurcated partition, the video decoding device 1900 may determine a third codec unit whose depth is increased by 1 compared to the depth of the first codec unit and two fourth codec units whose depths are each increased by 2 compared to the depth of the first codec unit. For example, when the depth of the first codec unit is N, the depth of the third codec unit may be N+1, and the depth of each of the two fourth codec units may be N+2, and the third codec unit and the fourth codec unit are generated by performing a trifurcated partition on the first codec unit. In this regard, the area of ​​the third codec unit may be 1 / 2 of the area of ​​the first codec unit, and the area of ​​each of the fourth codec units may be 1 / 4 of the area of ​​the first codec unit.

[0311] Hereinafter, a combination of pipeline data units that may be used by the video encoding apparatus 1700 and the video decoding apparatus 1900 will be described with reference to FIGS. 21 and 28 .

[0312] FIG. 21 shows the relationship between a maximum codec unit, a pipeline data unit, and a block according to an embodiment.

[0313] The partitioning method can be defined by considering the hardware design in the video encoding method that performs partitioning on the codec unit using quadtree partitioning, binary partitioning, and ternary partitioning. For example, in the conventional video codec standard, the size of the maximum codec unit (coding tree unit (CTU)) is 64x64, and therefore, the size of the pipeline data unit is also set to 64x64. However, in some codecs that are currently being standardized, the size of the maximum codec unit (CTU) 2100 can be determined to be up to 128x128. This means that the memory size or cache size is increased compared to the conventional codec standard. Therefore, in some codecs that are currently being standardized, the block partitioning rule is limited to a pipeline data unit 2110 having a size of 64x64. Specifically, constraints have been set that do not allow ternary partitioning in blocks of size greater than 64x64 (width>64 or height>64), and only allow binary partitioning in a ratio of 1:2 or 2:1 in blocks of size greater than 64x64 (i.e., only blocks of size 128x128, 128x64, and 64x128 are allowed. Because the current maximum transform size is 64 (i.e., the size of the maximum transform unit is 64x64), it may be allowed to split a block into blocks larger than the pipeline data unit 2110 of size 64x64.

[0314] In addition, in order to complete the encoding or decoding process of the codec unit during the pipeline cycle of the pipeline data unit 2110, the codec unit needs to be located in the pipeline data unit 2110 without crossing the boundary line (hereinafter referred to as the pipeline boundary line) of the pipeline data unit 2110. Although the codec units 2120 and 2130 are located in the pipeline data unit 2110, the codec units 2140 and 2150 cross the pipeline boundary line, so the encoding or decoding process is difficult to complete within one pipeline cycle.

[0315] According to an embodiment, in order to complete the transformation / inverse transformation performed based on the transformation unit during the pipeline cycle, it is preferred that the transformation unit is located in the pipeline data unit 2110. In particular, the transformation unit is not allowed to cross the pipeline boundary line. Therefore, according to an embodiment, the video encoding device 1700 and the video decoding device 1900 may prohibit the determination of the transformation unit from the codec unit. To this end, the video encoding device 1700 and the video decoding device 1900 may determine the size of the maximum transformation unit to 64x64, which is the size of the pipeline data unit and is less than or equal to the size of the codec unit.

[0316] According to an embodiment, the video encoding device 1700 and the video decoding device 1900 can determine the transform unit that does not cross the pipeline boundary line by dividing the transform unit that crosses the pipeline boundary line into smaller sizes. However, there is a problem that the division of the transform unit needs to be changed according to the position of the pipeline boundary line and the codec unit and the shape of the codec unit.

[0317] According to another embodiment, when the codec unit crosses the pipeline boundary line, the video encoding device 1700 may not be able to perform a transform on the corresponding block within the pipeline cycle, and thus may be able to encode the block without residual data. That is, because the codec unit is a codec unit that does not include residual data, there is no need to determine the transform unit.

[0318] According to an embodiment, the video encoding device 1700 may set the encoding mode of the codec unit crossing the pipeline boundary line to the skip mode, and encode the codec unit without encoding the residual data. In response to this, the video decoding device 1900 may determine the encoding mode of the codec unit crossing the pipeline boundary line as the skip mode, and decode the codec unit without decoding the residual data. In addition, the encoding mode of the codec unit crossing the pipeline boundary line is limited to the skip mode, and other encoding modes may be prohibited.

[0319] According to another embodiment, even if the coding mode of the codec unit crossing the pipeline boundary line is a mode other than the skip mode, the video encoding device 1700 may encode the codec unit without residual data by setting the coded block flag (cbf) of the codec unit or the transform unit to 0. In response to this, the video decoding device 1900 may determine that the coding mode of the codec unit crossing the pipeline boundary line has a cbf of 0 regardless of the coding mode, and decode the codec unit without decoding the residual data.

[0320] Therefore, the video encoding device 1700 may set the encoding mode of the block (codec unit or transform unit) crossing the pipeline boundary line to a mode that can be processed without residual data, so as to encode the corresponding block (codec unit or transform unit). Similarly, the video decoding device 1900 may determine the encoding mode of the block (codec unit or transform unit) crossing the pipeline boundary line to a mode that can be processed without residual data, so as to decode the corresponding block (codec unit or transform unit).

[0321] In addition, in the sub-block based prediction method, the entire block is set as the motion vector estimation range, so the video encoding device 1700 and the video decoding device 1900 may not be able to apply the sub-block based prediction method to the codec unit crossing the pipeline boundary line. Therefore, the sub-block based prediction method (such as the affine skip method) is not applicable to the codec unit crossing the pipeline boundary line.

[0322] However, as an exception, the problem caused by the pipeline method does not occur in the largest codec unit that crosses the outline of the image. Therefore, the restriction of prohibiting residual data at the pipeline boundary may not be applied to the largest codec unit that crosses the outline of the image.

[0323] In Figure 21, only the case where the pipeline data unit size is 64x64 is described above. However, there is a disadvantage that the encoding performance is reduced by limiting the pipeline data unit to only square blocks. Therefore, below in Figures 22 to 28, a method for improving encoding performance by allowing pipeline data units of various shapes (such as square and non-square) but keeping the size (area) of the pipeline data unit at 64×64=4096 is provided. Because the various shapes of pipeline data units require operations for memory addressing, the complexity of the implementation may increase, but the encoding performance can be increased. Therefore, when selecting the pipeline type, the optimal pipeline data unit can be selected considering the encoding performance and the amount of operation. An embodiment of the allowed pipeline data unit combination will be shown in Figures 22 to 28 below.

[0324] FIG. 22 shows three shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0325] As shown in Figure 22, the allowed pipeline data unit combination may include pipeline data units of three shapes. That is, the maximum codec unit of 128x128 size may include a combination 2210 of four pipeline data units of 64x64 size, a combination 2220 of four pipeline data units of 32x128 size, and a combination 2230 of four pipeline data units of 128x32 size. In this case, for example, three-way partitioning may not be allowed in blocks of 128x128, 128x64, and 64x128 sizes.

[0326] When a 128x128 block is vertically split and its first 64x128 block is vertically binary split, its second 64x128 block can be vertically binary split unconditionally (horizontal binary split and no split may not be allowed in the second 64x128 block). When the first 64x128 block is not split or is horizontally binary split, vertical binary split may not be allowed in the second 64x128 block.

[0327] In addition, when a 128x128-sized block is horizontally split and its first 128x64-sized block is horizontally binary split, its second 128x64-sized block can be horizontally binary split unconditionally (vertical binary split and no split may not be allowed in the second 128x64-sized block). When the first 128x64-sized block is not split or is vertically binary split, horizontal binary split may not be allowed in the second 128x64-sized block.

[0328] FIG. 23 shows five shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0329] As shown in Figure 23, the allowed pipeline data unit combination can include pipeline data units of five shapes. In this case, for example, it can be considered to change the constraints in the embodiment of Figure 22 so that trifurcated partitioning is allowed in a block of 128x128 size. In this case, when a block of 128x128 size is trifurcated and its central block is binary partitioned, binary partitioning in the same direction as the trifurcated partitioning can be allowed in the central block. When allowed, combinations 2320 and 2330 of four rectangular pipeline data units of the same size can be determined.

[0330] However, when the 128x128 size block is trifurcated and the center block is binary divided, binary division is not allowed in the center block in the same direction as the trifurcated division direction. In this case, the pipeline data unit of combination 2340 or 2350 in FIG. 23 may be used.

[0331] FIG. 24 shows seven shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0332] As shown in Fig. 24, the allowed pipeline data unit combination may include pipeline data units of seven shapes. According to an embodiment, when trifurcated partitioning is not allowed in blocks of 128x128, 128x64, and 64x128 sizes, the combination of Fig. 24 may be determined.

[0333] FIG. 25 shows nine shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to another embodiment.

[0334] As shown in Figure 25, the allowed pipeline data unit combination may include pipeline data units of nine shapes. According to an embodiment, when three-way partitioning is allowed in a block of 128x128 size and three-way partitioning is not allowed in a block of 128x64 size and a block of 64x128 size, the combination of Figure 25 may be determined.

[0335] FIG. 26 shows two shapes of combinations of pipeline data units that may be determined from a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0336] As shown in FIG26, the allowed pipeline data unit combination may include pipeline data units of two shapes. According to an embodiment, when trifurcated partitioning is not allowed in blocks of 128x128, 128x64, and 64x128 sizes and only vertical binary partitioning is allowed in blocks of 128x128 size, the combination of FIG26 may be determined. When a block of 128x128 size is vertically partitioned and its first block of 64x128 size is vertically binary partitioned, its second block of 64x128 size may be vertically binary partitioned unconditionally. Horizontal binary partitioning and no partitioning are not allowed in the second block of 64x128 size.

[0337] FIG. 27 shows two shapes of combinations of pipeline data units that can be determined only through quadtree partitioning or binary partitioning of a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0338] As shown in FIG27 , the allowed pipeline data unit combination may include pipeline data units of two shapes. For example, ternary partitioning may not be allowed in blocks of 128x128, 128x64, and 64x128 sizes, and only horizontal binary partitioning is allowed in blocks of 128x128 size. When a block of 128x128 size is horizontally partitioned and its first block of 128x64 size is horizontally binary partitioned, its second block of 128x64 size may be horizontally binary partitioned unconditionally (vertical binary partitioning and no partitioning may not be allowed in the second block of 128x64 size).

[0339] 28 shows three shapes of combinations of pipeline data units that may be determined by quadtree partitioning, ternary partitioning, and binary partitioning of a maximum codec unit in consideration of the size of the pipeline data unit according to an embodiment.

[0340] As shown in FIG. 28, the allowed combinations of pipeline data units may include pipeline data units of three shapes. When a ternary division is allowed in a 128x128-sized block, and thus the 128x128-sized block is ternary divided and its central block is binary divided, a binary division in the same direction as the ternary division direction may not be allowed in the central block. In this case, depending on whether the ternary division and the division direction are allowed, a combination of pipeline data units can be selected from three combinations.

[0341] The various shapes of the pipeline data units have been described above with reference to FIGS. 21 to 28. However, in addition to the above embodiments, the pipeline data units can be configured in any combination that can be easily modified by those of ordinary skill in the art.

[0342] Although the shapes of the allowed pipeline data units vary according to the above various situations, it remains the same that the residual data of the blocks crossing the pipeline boundary line is not processed. Next, a method for determining whether the current block crosses the pipeline boundary line will be described with reference to FIG. 29.

[0343] FIG. 29 shows a function for determining whether a codec unit is located at the boundary line of a pipeline data unit according to an embodiment.

[0344] In the function of FIG. 29, the size of the pipeline data unit is 64x64, the coordinates of the upper left sample point of the current block are (x, y), and the height and width of the current block are h and w, respectively.

[0345] When the right boundary of the current block is outside the right boundary line of the pipeline data unit (if (left_pipeline_boundary + 64 < x + w)), it can be determined that the current block crosses the boundary line of the pipeline data unit (result = true).

[0346] When the lower boundary of the current block is outside the lower boundary line of the pipeline data unit (if (top_pipeline_boundary + 64 < y + h)), it can be determined that the current block crosses the boundary line of the pipeline data unit (result = true).

[0347] When the size of the current block is 128x128 (if (w == 128 && h == 128)), it can be determined that the current block does not cross the boundary line of the pipeline data unit (result = false).

[0348] When the width of the current block is 64 and the height of the current block is 128 (if (w == 64 && h == 128 && (w % 64) == 0)), it can be determined that the current block does not cross the boundary line of the pipeline data unit (result = false).

[0349] When the height of the current block is 64 and the width of the current block is 128 (if (h==64&&w==128&&(h%64)==0)), it can be determined that the current block does not cross the boundary line of the pipeline data unit (result=false).

[0350] When the right boundary or the bottom boundary of the current block is outside the outline of the image (if ((x+128)>pic_w|| (y+128)>pic_h)), it can be determined that the current block does not cross the boundary line of the pipeline data unit (result=false).

[0351] Hereinafter, various embodiments will be described in which the video encoding apparatus 1700 and the video decoding apparatus 1900 signal information about a maximum size, a minimum size, and a difference between the maximum size and the minimum size of various blocks used for video encoding and decoding.

[0352] The size of the maximum codec unit and the minimum size of the codec unit can be defined independently of the partitioning mode. Before reaching the minimum size of the codec unit, the codec unit can be recursively partitioned using partitioning methods of various partitioning types (quadtree partitioning, binary partitioning, and ternary partitioning types). That is, when performing recursive partitioning, it can be determined whether the current codec unit can be further divided into sub-codec units by comparing the size of the current codec unit with the minimum size of the codec unit. For example, when identifying whether to perform quadtree partitioning, when quadtree partitioning is performed on the current codec unit and the size of the sub-codec unit becomes smaller than the minimum size of the codec unit, quadtree partitioning may not be performed. As another example, when identifying whether to perform binary partitioning, when binary partitioning is performed on the current codec unit in a specific direction, when the length of the corresponding side of the sub-codec unit in the partitioning direction becomes smaller than the minimum size of the codec unit, binary partitioning in a specific direction may not be performed. As another example, when identifying whether to perform trifurcating, when trifurcating in a specific direction is performed on the current codec unit and the length of the corresponding side of the sub-codec unit in the division direction becomes smaller than the minimum size of the codec unit, the trifurcating in the specific direction may not be performed.

[0353] In this regard, information about the minimum size of the codec unit may be predefined by the video encoding device 1700 and the video decoding device 1900. The video encoding device 1700 may encode the information about the minimum size of the codec unit and include the encoded information in a sequence header, SPS, picture header, picture parameter set (PPS), or slice header. In response, the video decoding device 1900 may obtain information about the minimum size of the codec unit from the sequence header, SPS, picture header, PPS, or slice header.

[0354] As a specific example, the minimum size of the codec unit may be 4. In addition, the value after log2 of the minimum size of the codec unit may be encoded. That is, when the minimum size of the codec unit is 4, the value corresponding to the log2 of the codec unit is 2, and when -2 is applied thereto, the resulting value may be 0. By applying -2 to the value after log2 of the minimum size of the codec unit and then signaling the resulting value, the encoding and decoding efficiency of the information on the minimum size of the codec unit may be improved. In this regard, a fixed length coding (FLC) method or a variable length coding (VLC) method may be used as a method for encoding the information on the minimum size of the codec unit. In this case, when the VLC method is used, an unsigned exponential Golomb code method may be used. As another example, when the minimum size of the codec unit is 8, the resulting value may be signaled by applying -2 to the value (3) corresponding to the log2 and setting the value to 1.

[0355] The allowed partition depth (indicating the number of recursive partitions allowed in the corresponding type) for various partition types (quadtree partition, binary partition, ternary partition type) can also be set separately for each partition type. When the video encoding device 1700 and the video decoding device 1900 support quadtree partition, binary partition and ternary partition, for example, the allowed partition depth for quadtree partition, the allowed partition depth for binary partition and the allowed partition depth for ternary partition can be set. As another example, the allowed partition depth for quadtree partition and the allowed partition depth for binary partition and ternary partition can also be defined. In this case, the allowed partition depth for binary partition and ternary partition can be set equally. As another example, the allowed partition depth for quadtree partition, binary partition and ternary partition can be set to a single partition depth.

[0356] In the aforementioned embodiment of presetting the partition depth, the concept of depth can be considered as the concept of area. For example, when a block is partitioned and therefore the area of ​​its sub-block is 1 / N, the depth of the sub-block can be set to be 1 more than the depth of the block. For example, in the case of N=2, when two M / 2xM blocks are determined by performing binary partitioning on an MxM block (depth=0), the area of ​​each M / 2xM block is 1 / N (i.e., 1 / 2), so that the depth of each M / 2xM block increases by 1, and thus the depth can be 1. In addition, when four M / 2xM / 2 blocks are determined by performing quadtree partitioning on an MxM block (depth=0), the area of ​​each M / 2xM / 2 block is 1 / N×1 / N (i.e., 1 / 2×1 / 2=1 / 4), so that the depth of each M / 2xM / 2 increases by 2, and thus the depth can be determined to be 2. Furthermore, when two M / 4xM blocks and one M / 2xM block are determined by performing trifurcated partitioning on an MxM block (depth=0), the depth of each M / 4xM block increases by 2, so the depth may be determined as 2, and the depth of the M / 2xM block increases by 1, so the depth may be determined as 1. Furthermore, depth information may be individually set depending on a partitioning method.

[0357] In addition, there is a module that performs a specific operation by comparing the depth information of the adjacent block with the depth information of the current block, and the depth configuration method in the corresponding segmentation method can be used. For example, the context-based adaptive binary arithmetic code (CABAC) context can be determined by comparing the depth of the current block with the depth of the adjacent block. The embodiment is not limited to the depth of a specific partitioning pattern, and the depth of the current block and the depth of the adjacent block are compared based on the actual partitioning area or block size, so that a more accurate context can be determined.

[0358] Assuming that the allowed partition depths of binary partition and ternary partition are the same as each other, and the allowed partition depth is N, in the case of binary partition, when the current depth is D, the depth of each sub-block is D+1. However, in the case of ternary partition, since the areas of the sub-blocks are different from each other, the depths of the first sub-block and the third sub-block are determined to be D+2, and the depth of the second (middle) sub-block is determined to be D+1. In this regard, when the allowed partition depth (N) is D+3, an additional partition can be performed on the first sub-block and the third sub-block once, and two additional partitions can be performed on the second sub-block. As another example, an additional partition can be performed on the first sub-block and the third sub-block once, but only a partition mode that increases the depth of the sub-block by 1 is allowed. That is, because there is a partition mode that increases the depth of the sub-block by 2 when the ternary partition is performed on the corresponding sub-block, the ternary partition may not be used, and a partition mode that increases the depth of the sub-block by 1 (such as binary partition) may be allowed. That is, the depth of the block as a leaf node can be set to a value that does not exceed the partition depth.

[0359] According to an embodiment, the video encoding device 1700 and the video decoding device 1900 may only pre-set the maximum size and the minimum size of the codec unit, and may not signal the allowed partition depth. In this case, the partition depth allowed by quadtree partitioning, binary partitioning, and ternary partitioning reaches the minimum size of the codec unit, and no signaling is given for the specifically allowed partition depth. However, even in this case, information about the partition depth may be stored in a storage device for execution of other modules.

[0360] In a partitioning structure that allows binary partitioning and trifurcating partitioning, when a constraint is imposed on the maximum size of a codec unit for each of the binary partitioning and the trifurcating partitioning (for example, when the binary partitioning or the trifurcating partitioning is not performed with a size larger than the maximum size of the codec unit), the value of the maximum size of the codec unit for the trifurcating partitioning may be set to be always equal to or less than the value of the maximum size of the codec unit for the binary partitioning. As a specific example, when the value of the maximum size of the codec unit for the binary partitioning is 64, the value of the maximum size of the codec unit for the trifurcating partitioning may be equal to or less than 64, i.e., 32.

[0361] According to an embodiment, the video encoding device 1700 and the video decoding device 1900 may allow only binary partitioning and trifurcating partitioning in a codec unit having a fixed size or smaller by presetting the maximum size of the codec unit for binary partitioning and trifurcating partitioning. As a specific example, binary partitioning and trifurcating partitioning may always be allowed only on edges having a length of 64 or less.

[0362] According to an embodiment, the video encoding device 1700 and the video decoding device 1900 may use the maximum transform size as a reference when designing a hardware pipeline. In this case, a constraint may be imposed on the partitioning so that the number of blocks that are larger than the maximum transform size relative to the longer side of the block does not exceed (the maximum size of the codec unit) / (the maximum transform size) within the maximum codec unit. For example, when the maximum size of the codec unit is 128 and the maximum transform size is 64, in the maximum codec unit of size 128x128, 128 / 64 blocks (i.e., a maximum of two blocks) may be allowed, each of the two blocks having at least two sides of length 128 greater than 64. Here, because a block having a side length of 128 has a maximum transform size of 64, the transform should be performed by dividing one side of the block having a length of 128. That is, because a block having a side length of 128 includes two transform blocks having the maximum transform size, two pipeline data units are required for pipeline design, so a maximum of two blocks may be allowed by default. Specifically, when trifurcating partitioning is performed on the maximum codec unit of size 128x128 at a ratio of 1:2:1, three blocks having a side length of 128 are generated, and thus such trifurcating partitioning may not be permitted. Alternatively, when horizontal binary partitioning is performed again on each sub-block (128x64) generated by performing vertical binary partitioning on the maximum codec unit of size 128x128 at a ratio of 1:1, three or more blocks having a side length of 128 are generated, and thus binary partitioning of these sub-blocks is not permitted.

[0363] In a block structure that performs partitioning using quadtree partitioning, binary partitioning, and ternary partitioning, relative to the maximum transform size, ternary partitioning may not be allowed for blocks with sides larger than the maximum transform size, and only ternary partitioning may be allowed for blocks with aspect ratios of 1:1, 1:2, and 2:1. As a specific example, when the maximum transform size is 64, ternary partitioning is not allowed by default for blocks of 128x128 size, and only ternary partitioning is allowed for blocks of 1:1, 1:2, and 2:1 aspect ratios. Therefore, because the binary partitioned sub-blocks (blocks with aspect ratios of 1:2 or 2:1, i.e., blocks of 128x64 size or 64x128 size) already have a ratio of 1:2 or 2:1, binary partitioning that partitions the longer side may be allowed.

[0364] In a structure that allows the block generated by quadtree partitioning to be split using binary partitioning and ternary partitioning, the partition depth allowed by binary partitioning and ternary partitioning can be defined. In this case, the partition depth allowed by binary partitioning and ternary partitioning can be defined depending on the depth of quadtree partitioning (or the block size of the block generated by quadtree partitioning). For example, when the depth of the block generated by quadtree partitioning is between 0 and 2, the partition depth allowed by binary partitioning and ternary partitioning in the block generated by quadtree partitioning can be determined to be 2. When the depth of the block generated by quadtree partitioning is between 3 and the last depth or the minimum size allowed by the block generated by quadtree partitioning, the partition depth allowed by binary partitioning and ternary partitioning can be set to 3.

[0365] As another example, when the depth of the block generated by quadtree partitioning is between 0 and 2, the partition depth allowed by binary partitioning and ternary partitioning may be set to 3. When the depth of the block generated by quadtree partitioning is between 3 and the last depth or the minimum size allowed by the block generated by quadtree partitioning, the partition depth allowed by binary partitioning and ternary partitioning may be set to 2.

[0366] In a method of independently segmenting a luminance block and a chrominance block, segmentation information about the luminance block and segmentation information about the chrominance block may be transmitted separately. In the above method, the video decoding apparatus 1900 may apply parameters for determining a segmentation method of the luminance block to a segmentation method of the chrominance block in the same manner. As another example, parameters for segmentation methods of the luminance block and the chrominance block are signaled separately so that different segmentation methods may be performed.

[0367] Below, a specific example with respect to the size of the picture sample unit will be given. For the luminance block and the chrominance block, the maximum size of the binary divisible codec unit can be determined to be the same. In this regard, the maximum size of the binary divisible codec unit can be a value pre-agreed by the video encoding device 1700 and the video decoding device 1900, or can be a value notified by header signaling. The value determined in units of sequences can be used together in intra-frame slices and inter-frame slices, or the value determined in units of slices can be used together by signaling for each slice.

[0368] As another example, for each luminance block and each chrominance block, the maximum size of the binary divisible codec unit may be determined differently. In this regard, the maximum size of the binary divisible codec unit may be a value agreed upon in advance by the video encoding device 1700 and the video decoding device 1900, or may be a value notified through header signaling. In this case, the maximum size of the binary divisible chrominance block may be greater than the maximum size of the binary divisible luminance block in units of picture samples.

[0369] As another example, in a method of independently segmenting a luminance block and a chrominance block, when the maximum size of a binary divisible codec unit is determined in units of a signaled picture sample, the maximum size may be interpreted based on the actual luminance block and the chrominance block. For example, when a 4:2:0 YUV color format is used and the maximum size of a binary divisible codec unit is determined to be 32, the maximum size of the binary divisible codec unit may be applied in units of luminance samples in the segmentation of the luminance block, and the maximum size of the binary divisible codec unit may be applied in units of chrominance samples in the segmentation of the chrominance block. In this case, in the segmentation of the chrominance block, a parameter corresponding to a size twice the width and height of the luminance block relative to the picture sample is applied.

[0370] In the above-described embodiments, the video encoding apparatus 1700 and the video decoding apparatus 1900 may perform signaling on the maximum size of a binary-dividable codec unit and the maximum size of a tri-dividable codec unit in an integrated manner.

[0371] The following are examples of block segmentation methods that may be used by the video encoding apparatus 1700 and the video decoding apparatus 1900 .

[0372] - In a method of independently partitioning a luminance block and a chrominance block, parameters of the partitioning method of the luminance block and the chrominance block may be signaled in an integrated manner for the luminance block and the chrominance block, and the partitioning method may also be shared between the luminance block and the chrominance block.

[0373] -Parameters of the partitioning method in the intra slice and the inter slice can be integrated and then signaled, and the partitioning method can be shared between the intra slice and the inter slice. By setting the parameters in units of a sequence, the corresponding partitioning method can be shared in all pictures.

[0374] -When the maximum size of the codec unit is 128x128, the block with the maximum size (binary division is allowed) can be fixedly determined as a codec unit of 64x64 size in the intra-frame slice. In addition, the block with the maximum size that can be divided three-way can be determined as a 64x64 codec unit in the same way as the binary division, or can be determined as a codec unit of 32x32 size (which corresponds to 1 / 4 of 128x128). The block with the maximum size (binary division is allowed) can be fixedly determined as a codec unit of 128x128 size in the inter-frame slice. In addition, the block with the maximum size that can be divided three-way can be determined as a 128x128 codec unit in the same way as the binary division, or can be determined as a codec unit of 64x64 size (which corresponds to 1 / 4 of 128x128).

[0375] - The maximum size of the trifurcated codec unit may be determined based on the maximum size of the binary divisible codec unit. For example, a size corresponding to 1 / 4 of the maximum size of the binary divisible codec unit may be determined as the maximum size of the trifurcated codec unit. When the maximum size of the binary divisible codec unit is MxN, the maximum size of the trifurcated codec unit may be M / 2xN / 2. That is, the maximum size of the trifurcated codec unit may be determined based on the signaling notification of the maximum size of the binary divisible codec unit.

[0376] -According to an embodiment, the video encoding device 1700 and the video decoding device 1900 may fixedly set the maximum size of the binary divisible codec unit and the maximum size of the trifurcated codec unit without signaling. When a method of independently dividing a luminance block and a chrominance block is applied, the maximum size of the binary divisible block and the maximum size of the trifurcated block may also be fixedly set depending on the slice type. In addition, the maximum size of the trifurcated block may be determined to be equal to the maximum transform size. As a specific example, the maximum size of the binary divisible block may be set differently for an intra slice and an inter slice, but the maximum size of the binary divisible block in the intra slice may be set to be smaller than the maximum size of the binary divisible block in the inter slice.

[0377] In addition, when a method of independently dividing the luma block and the chroma block is applied, the maximum size of the binary divisible luma block can be set to be smaller than the maximum size of the binary divisible chroma block in the same slice. Similarly, the maximum size of the trifurcated luma block can be set to be smaller than the maximum size of the trifurcated chroma block in the same slice. As a specific example, the maximum size of the binary divisible block in the intra slice can be set to 32, and the maximum size of the binary divisible block in the inter slice can be set to 128.

[0378] When a method of independently partitioning luminance blocks and chrominance blocks is applied, the maximum size of a binary divisible chrominance codec unit in an intra slice may be set to 64. In addition, for a codec unit having a side length of 128, only partitioning at a ratio of 1:2 or 2:1 is allowed.

[0379] -For example, the video decoding device 1900 may obtain the following parameters from a bitstream, and the video encoding device 1700 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0380] * Maximum size of the codec unit

[0381] * Maximum size of binary divisible codec unit (applicable to luma block and chroma block) (When a method of independently partitioning luma block and chroma block is applied, information about the maximum size of binary divisible codec unit can be shared and used in luma block and chroma block.)

[0382] * Maximum size of a three-way divisible codec unit (applicable to luma blocks and chroma blocks) (When a method of independently splitting luma blocks and chroma blocks is applied, information about the maximum size of a three-way divisible codec unit can be shared and used in luma blocks and chroma blocks.)

[0383] - As another example, the video decoding device 1900 may obtain the following parameters from a bitstream, and the video decoding device 1900 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0384] * Maximum size of the codec unit

[0385] * Maximum size of a binary-divisible codec unit and a tri-divisible codec unit (applicable to luma blocks and chroma blocks) (when a method of independently splitting luma blocks and chroma blocks is applied, information about the maximum size of a binary-divisible codec unit can be shared and used in luma blocks and chroma blocks).

[0386] - In addition, as another example, the video decoding device 1900 may obtain the following parameters from the bitstream, and the video decoding device 1900 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0387] * Maximum size of the codec unit

[0388] * Maximum size of binary divisible codec unit (applicable to luma blocks and chroma blocks) (The maximum size of a trifurcating codec unit can be determined by using the maximum size of a binary divisible codec unit.)

[0389] - In addition, as another example, the video decoding device 1900 may obtain the following parameters from the bitstream, and the video decoding device 1900 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0390] * Maximum size of the codec unit

[0391] * Maximum size of binary and trifurcated luma codec units

[0392] * Maximum size of binary-divisible and tri-divisible chroma codec units when applying the method of independently partitioning luminance blocks and chroma blocks

[0393] *When the method of independently partitioning the luma block and the chroma block is not applied, the maximum size of the binary-divisible and tri-divisible luma codec unit may be applied to the partitioning of the chroma codec unit.

[0394] - In addition, as another example, the video decoding device 1900 may obtain the following parameters from the bitstream, and the video decoding device 1900 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0395] * Maximum size of the codec unit

[0396] * Maximum size of binary divisible luma codec unit

[0397] * When the method of independently partitioning luminance blocks and chrominance blocks is applied, the maximum size of the chrominance codec unit that can be binary divided

[0398] *When the method of independently partitioning the luma block and the chroma block is not applied, the maximum size of the binary divisible luma codec unit may be applied to the partitioning of the chroma codec unit.

[0399] * Maximum size of a trifurcated luma codec unit

[0400] * When the method of independently partitioning luminance blocks and chrominance blocks is applied, the maximum size of the chrominance codec unit that can be divided into three parts

[0401] *When the method of independently partitioning the luma block and the chroma block is not applied, the maximum size of the tripartite-divisible luma codec unit may be applied to the partitioning of the chroma codec unit.

[0402] - In addition, as another example, the video decoding device 1900 may obtain the following parameters from the bitstream, and the video decoding device 1900 may encode the following parameters and then transmit the encoded parameters included in the bitstream.

[0403] * Maximum size of the codec unit

[0404] * Maximum size of a binary divisible luma codec unit in an intra slice

[0405] * When the method of independently partitioning luma blocks and chroma blocks in intra slices is applied, the maximum size of the chroma codec unit that can be binary divided

[0406] * Maximum size of a luma codec unit that can be trifurcated in an intra slice

[0407] * When the method of independently partitioning luma blocks and chroma blocks in intra slices is applied, the maximum size of the chroma codec unit that can be trifurcated

[0408] * Maximum size of binary divisible luma codec unit in inter-frame slices

[0409] * When the method of independently partitioning luma blocks and chroma blocks in inter slices is applied, the maximum size of the chroma codec unit that can be binary divided

[0410] * Maximum size of a trifurcated luma codec unit in an inter-frame slice

[0411] * When the method of independently partitioning luma blocks and chroma blocks in inter slices is applied, the maximum size of the chroma codec unit that can be trifurcated

[0412] - In addition to the above embodiments, signaling of the minimum size of the coding unit may also be performed.

[0413] *When there are various partitioning methods, the minimum size of the codec unit is not defined for each division mode, but the minimum size of the codec unit may be defined and signaled, and the minimum size of the codec unit is generally used for various division modes. Specifically, the minimum size of the codec unit may be determined as MxN, such as 4x4, etc. When M and N are the same as each other, only information about M may be signaled, and when M and N are different from each other, the value of M and the difference between M and N may be signaled (or the value of N and the difference between N and M may be signaled).

[0414] * In addition, the definition of the minimum size of the codec unit can be set for each inter block and each intra block. For example, the minimum size of the codec unit for the intra block (e.g., min_cu_size_intra), the value obtained by applying log2 to the minimum size of the codec unit (e.g., log2_min_cu_size_intra), or the value obtained by applying -2 to the value after log2 (e.g., log2_min_cu_size_intra_minus2) can be signaled. In addition, the inter block is similar to the intra block, and the minimum size of the codec unit (e.g., min_cu_size_inter), the value obtained by applying log2 to the minimum size of the codec unit (e.g., log2_min_cu_size_inter), or the value obtained by applying -2 to the value after log2 (e.g., log2_min_cu_size_inter_minus2) can be signaled.

[0415] * In addition, for each partitioning mode, information about the minimum size of the codec unit can be signaled. For example, information about the minimum size of the codec unit allowed by quadtree partitioning, binary partitioning, and ternary partitioning can be signaled separately, and the minimum size of the codec unit allowed by quadtree partitioning and the minimum size of the codec unit allowed by binary partitioning and ternary partitioning can be signaled.

[0416] - In addition to the above embodiments, additional allowed partition depths may be defined for each partition mode. The allowed partition depths for quadtree partitioning (e.g., max_qp_depth), the allowed partition depths for ternary partitioning (e.g., max_tt_depth), and the allowed partition depths for binary partitioning (e.g., max_bt_depth) may be set, and then these partition depths may be signaled. As another example, the allowed partition depths for binary partitioning and ternary partitioning (e.g., max_btt_depth) may be set, and then the partition depths may be signaled.

[0417] -As a specific embodiment, the size of the binary-dividable codec unit in the intra slice may be configured as 64 (shared between the luma block and the chroma block), the size of the trifurcated codec unit in the intra slice may be configured as 32 (shared between the luma block and the chroma block), the size of the binary-dividable codec unit in the inter slice may be configured as 128, and the size of the trifurcated codec unit in the inter slice may be configured as 64.

[0418] As a specific embodiment, the maximum size of the binary-dividable codec unit and the ternary-dividable codec unit in the intra slice may be configured as 32, and the maximum size of the binary-dividable codec unit and the ternary-dividable codec unit in the inter slice may be configured as 64.

[0419] - As a specific embodiment, the maximum size of the binary-dividable codec unit and the tri-dividable codec unit in the intra slice may be configured as 32, the maximum size of the binary-dividable codec unit in the inter slice may be configured as 128, and the maximum size of the binary-dividable codec unit in the inter slice may be configured as 64. For codec units with sizes from 128x128 to 64x64, it may be configured to allow only blocks with an aspect ratio of 1:2 or 2:1.

[0420] According to an embodiment, pseudo codes to be used by the video encoding device 1700 and the video decoding device 1900 to implement the above-mentioned embodiment in a pipeline structure are shown below. In the pseudo codes shown in Table 1 below, the partition modes BT, TT and QT indicate binary partition, ternary partition and quadtree partition, respectively.

[0421] [Table 1]

[0422] max_tt_size / / The maximum size of the codec unit that can be divided into three parts

[0423] max_bt_size / / The maximum size of the binary divisible codec unit

[0424] width / / The width of the current codec unit

[0425] height / / The height of the current encoding and decoding unit

[0426] log2_width / / log2 (width)

[0427] log2_height / / log2 (height)

[0428] / / (1. Test for trifurcated division in horizontal direction)

[0429] if (height > max_tt_size)

[0430] TT in horizontal direction is not allowed

[0431] else

[0432] TT in horizontal direction may be allowed / / There may be additional conditions to determine whether trifurcations in the horizontal direction are allowed.

[0433] / / (2. Test for the vertical trisection)

[0434] if (width > max_tt_size)

[0435] TT in vertical direction is not allowed

[0436] else

[0437] TT in vertical direction may be allowed / / There may be additional conditions to determine whether trifurcations in the vertical direction are allowed.

[0438] / / (3. Test for binary partition in horizontal direction)

[0439] if (abs (log2_height / 2 - log2_width) > 1 && (height > 64 || width >64))

[0440] BT in horizontal direction is not allowed

[0441] else

[0442] BT in horizontal direction is allowed / / There can be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0443] / / (4. Test for binary partition in the vertical direction)

[0444] if (abs (log2_width / 2 - log2_height) > 1 && (height > 64 || width >64))

[0445] BT in vertical direction is not allowed

[0446] else

[0447] BT in vertical direction is allowed / / There can be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0448] / / End of table 1

[0449] As another embodiment, Test 1 and Test 2 may be as shown in Table 2. It may be set so that only trifurcated partitioning is allowed in the case of a size of the codec unit (max_tt_size x max_tt_size) or smaller. As an example, in Test 1, even if the width of the current codec unit is greater than max_tt_size, although the height of the codec unit is equal to or less than max_tt_size, the corresponding partitioning mode may be allowed. However, in the case of Table 2 below, even if another side is greater than max_tt_size, the corresponding partitioning mode may not be allowed.

[0450] [Table 2]

[0451] / / (1. Test for trifurcated division in horizontal direction)

[0452] if (height > max_tt_size || width > max_tt_size)

[0453] TT in horizontal direction is not allowed

[0454] else

[0455] TT in horizontal direction may be allowed / / There may be additional conditions to determine whether trifurcations in the horizontal direction are allowed.

[0456] / / (2. Test for the vertical trisection)

[0457] if (width > max_tt_size || height > max_tt_size)

[0458] TT in vertical direction is not allowed

[0459] else

[0460] TT in vertical direction may be allowed / / There may be additional conditions to determine whether trifurcations in the vertical direction are allowed.

[0461] / / End of table 2

[0462] As another embodiment, Test 1 and Test 2 may be as shown in Table 3. When the maximum size max_tt_size of the trifurcated codec unit is less than N, it may be set so that trifurcated division TT is allowed only when the size of the codec unit is (N×N) or smaller. As a specific example, N may be 64. In addition, in this case, the value of max_tt_size may be set to be less than or equal to N. As another example, in this regard, N may be set to the maximum transform size.

[0463] [Table 3]

[0464] / / (1. Test for trifurcated division in horizontal direction)

[0465] if (height > max_tt_size || width > N)

[0466] TT in horizontal direction is not allowed

[0467] else

[0468] TT in horizontal direction may be allowed / / There may be additional conditions to determine whether trifurcations in the horizontal direction are allowed.

[0469] / / (2. Test for the vertical trisection)

[0470] if (width > max_tt_size || height > N)

[0471] TT in vertical direction is not allowed

[0472] else

[0473] TT in vertical direction may be allowed / / There may be additional conditions to determine whether trifurcations in the vertical direction are allowed.

[0474] / / End of table 3

[0475] As another embodiment, Test 3 and Test 4 may be as shown in Table 4.

[0476] [Table 4]

[0477] / / (3. Test for binary partition in horizontal direction)

[0478] if(width > height && width > 64)

[0479] BT in horizontal direction is not allowed

[0480] else

[0481] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0482] / / (4. Test for binary partition in the vertical direction)

[0483] if(height > width && height > 64)

[0484] BT in vertical direction is not allowed

[0485] else

[0486] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0487] / / End of table 4

[0488] As another embodiment, Test 3 and Test 4 may be as shown in Table 5.

[0489] [Table 5]

[0490] / / (3. Test for binary partition in horizontal direction)

[0491] if (width > 64 && width > height)

[0492] BT in horizontal direction is not allowed

[0493] else

[0494] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0495] / / (4. Test for binary partition in the vertical direction)

[0496] if (height > 64 && height > width)

[0497] BT in vertical direction is not allowed

[0498] else

[0499] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0500] / / End of table 5

[0501] As another example, Tests 3 and 4 may be as shown in Table 6.

[0502] [Table 6]

[0503] / / (3. Test for binary partition in horizontal direction)

[0504] if (width > 64 && log2_width - log2_height == 1)

[0505] BT in horizontal direction is not allowed

[0506] else

[0507] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0508] / / (4. Test for binary partition in the vertical direction)

[0509] if (height > 64 && log2_height - log2_width == 1)

[0510] BT in vertical direction is not allowed

[0511] else

[0512] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0513] / / End of table 6

[0514] As another example, Tests 3 and 4 may be as shown in Table 7.

[0515] [Table 7]

[0516] / / (3. Test for binary partition in horizontal direction)

[0517] if (width > 64 && width != height)

[0518] BT in horizontal direction is not allowed

[0519] else

[0520] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0521] / / (4. Test for binary partition in the vertical direction)

[0522] if (height > 64 && height != width)

[0523] BT in vertical direction is not allowed

[0524] else

[0525] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0526] / / End of Table 7

[0527] As another example, Tests 3 and 4 may be as shown in Table 8.

[0528] [Table 8]

[0529] / / (3. Test for binary partition in horizontal direction)

[0530] if (width > 64 && width - height != 0)

[0531] BT in horizontal direction is not allowed

[0532] else

[0533] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0534] / / (4. Test for binary partition in the vertical direction)

[0535] if (height > 64 && height - width != 0)

[0536] BT in vertical direction is not allowed

[0537] else

[0538] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0539] / / End of table 8

[0540] As another embodiment, Test 3 and Test 4 may be as shown in Table 9. In the following embodiments, maximumtransform size indicates the maximum transform size.

[0541] [Table 9]

[0542] / / (3. Test for binary partition in horizontal direction)

[0543] if(height == 64 && width == 128)

[0544] BT in horizontal direction is not allowed

[0545] else

[0546] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0547] / / (4. Test for binary partition in the vertical direction)

[0548] if (height == 128 && width == 64)

[0549] BT in vertical direction is not allowed

[0550] else

[0551] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0552] / / End of table 9

[0553] As another example, Tests 3 and 4 may be as shown in Table 10.

[0554] [Table 10]

[0555] / / (3. Test for binary partition in horizontal direction)

[0556] if (height == maximum transform size && width == CTU size)

[0557] BT in horizontal direction is not allowed

[0558] else

[0559] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0560] / / (4. Test for binary partition in the vertical direction)

[0561] if (height == CTU size && width == maximum transform size)

[0562] BT in vertical direction is not allowed

[0563] else

[0564] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0565] / / End of table 10

[0566] As another example, Tests 3 and 4 may be as shown in Table 11.

[0567] [Table 11]

[0568] / / (3. Test for binary partition in horizontal direction)

[0569] if (height == maximum transform size && width > maximum transformsize)

[0570] BT in horizontal direction is not allowed

[0571] else

[0572] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0573] / / (4. Test for binary partition in the vertical direction)

[0574] if (height > maximum transform size && width == maximum transformsize)

[0575] BT in vertical direction is not allowed

[0576] else

[0577] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0578] / / End of table 11

[0579] As another example, Tests 3 and 4 may be as shown in Table 12.

[0580] [Table 12]

[0581] / / (3. Test for horizontal binary partition)

[0582] if (height == 64 && width > 64)

[0583] BT in horizontal direction is not allowed

[0584] else

[0585] BT in horizontal direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0586] / / (4. Test for binary partition in the vertical direction)

[0587] if(height > 64 && width == 64)

[0588] BT in vertical direction is not allowed

[0589] else

[0590] BT in vertical direction may be allowed / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0591] / / End of table 12

[0592] In the above embodiments (Table 1 to Table 12), it can be considered that various expression methods for designing pipeline structures with the maximum transform size of 64x64 as a unit have been provided.

[0593] Similar to the above embodiment, the method of dividing the codec unit at the outline of the image can be determined in combination with the design of the pipeline structure. When the codec unit crosses the outline of the image, the codec unit can be arbitrarily divided by using binary division, and as shown in the above embodiment, binary division can be allowed or prohibited under preset conditions. As another example, when quadtree division and binary division are allowed at the outline of the image at the same time but the preset conditions are not met, it can be set to use quadtree division. As a specific example, with respect to the codec unit that crosses the outline of the image, conditions 3 and 4 can be changed as shown in Table 13.

[0594] [Table 13]

[0595] / / (3. Test for binary partition in horizontal direction)

[0596] if (height == 64 && width > 64)

[0597] QT is used (use QT)

[0598] else

[0599] BT in horizontal direction may be used / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0600] / / (4. Test for binary partition in the vertical direction)

[0601] if(height > 64 && width == 64)

[0602] QT is used (use QT)

[0603] else

[0604] BT in vertical direction may be used / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0605] / / End of table 13

[0606] For reference, the division of the codec unit that crosses the outline of the image and the division of the codec unit that does not cross the outline of the image are as follows. For the codec unit that crosses the outline of the image, it can be considered whether to use the division mode, and for the codec unit that does not cross the outline of the image, it can be considered whether to allow the division mode. In another embodiment, when the current codec unit is determined by performing binary division on the codec unit of higher depth, after the binary division, the quadtree division of the current codec unit may not be allowed, and the binary division of the current codec unit in the same direction may also be limited. As an exception, the quadtree division of non-square codec units may be allowed. In addition, when the codec unit has been binary divided in another direction, because the binary division in a specific direction is difficult to perform, the division can be performed in a direction different from the division direction of the previously performed binary division. That is, when the binary division in the vertical direction has been performed because it is impossible to perform binary division in the horizontal direction, the binary division in the horizontal direction can be allowed subsequently. This can be represented by pseudo code, as shown in Table 14.

[0607] [Table 14]

[0608] / / (3. Test for binary partition in horizontal direction)

[0609] if (height == 64 && width > 64)

[0610] BT in vertical direction is used

[0611] else

[0612] BT in horizontal direction may be used (BT in horizontal direction may be used) / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0613] / / (4. Test for binary partition in the vertical direction)

[0614] if(height > 64 && width == 64)

[0615] BT in horizontal direction is used

[0616] else

[0617] BT in vertical direction may be used / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0618] / / End of table 14

[0619] In another embodiment, conditions 3 and 4 may be modified as shown in Table 15.

[0620] [Table 15]

[0621] / / (3. Test for binary partition in horizontal direction)

[0622] if (width != height && width > 64)

[0623] QT is used (use QT)

[0624] else

[0625] BT in horizontal direction may be used / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0626] / / (4. Test for binary partition in the vertical direction)

[0627] if (width != height && width == 64)

[0628] QT is used (use QT)

[0629] else

[0630] BT in vertical direction may be used / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed.

[0631] / / End of table 15

[0632] In another embodiment, for non-square codec units, binary partitioning may be allowed, as shown in Table 16.

[0633] [Table 16]

[0634] / / (3. Test for binary partition in horizontal direction)

[0635] if (width != height && width > 64)

[0636] BT in vertical is used

[0637] else

[0638] BT in horizontal direction may be used / / There may be additional conditions to determine whether binary partitioning in the horizontal direction is allowed.

[0639] / / (4. Test for binary partition in the vertical direction)

[0640] if (width != height && width == 64)

[0641] BT in horizontal is used

[0642] else

[0643] BT in vertical direction may be used (using BT in vertical direction) / / There may be additional conditions to determine whether binary partitioning in the vertical direction is allowed / /

[0644] / / End of table 16

[0645] The above-mentioned specific conditions for various division methods can be used together with various conditions for determining the division mode. In addition, when the maximum codec unit includes the outline of the image, the maximum codec unit is arbitrarily divided once using quadtree division, and the division method according to the division information can be applied to the codec unit generated by the quadtree division. As an example, when the size of the maximum codec unit is 128x128 and the maximum transform size is 64, the maximum codec unit is arbitrarily divided into four codec units of 64x64 size, and then the division method can be performed on the codec unit of 64x64 size according to the division information based on the division mode.

[0646] [Table 17]

[0647] if(width > 64 && height > 64)

[0648] QT split is used

[0649] / / End of table 17

[0650] As another example, when the maximum codec unit includes the right contour of the image, the maximum codec unit is vertically binary divided once, and when the codec unit generated by the vertical binary division requires additional division, only horizontal binary division may be allowed. Similarly, when the maximum codec unit includes the lower contour of the image, the maximum codec unit is horizontally binary divided once, and when the codec unit generated by the horizontal binary division requires additional division, only vertical binary division may be allowed.

[0651] According to an embodiment, when half or more of the codec unit located at the outline of the image is included in the image, additional division may be allowed in consideration of the number of pipeline modules allowed by the pipeline structure. For example, under the assumption that the number of pipeline modules in the codec unit of size 128x128 is 4, a case where the maximum transform size is 64 may be assumed. When the area of ​​the codec unit of size 64x128 is located within the outline of the image, binary division in the vertical direction may be allowed so as to allow the generation of two codec units of size 32x128. In this regard, additional binary division of the generated codec unit in the vertical direction may not be allowed.

[0652] Hereinafter, the SPS is modified based on the working draft 2.0 of the Versatile Video Coding (VVC) standard so that information on the size of a block limited in various partition modes can be signaled.

[0653] 30 shows a syntax notified by SPS signaling according to an embodiment. According to an embodiment, the video encoding device 1700 may encode information n about the size of the block limited in various partition modes, and include the encoded information in the SPS of FIG. 30. According to an embodiment, the video decoding device 1900 may obtain information about the size of the block limited in various partition modes from the SPS of FIG. 30.

[0654] Through SPS (seq_parameter_set_rbsp()), information about the maximum size of a binary divisible codec unit (log2_diff_ctu_max_bt_size_intra, log2_diff_ctu_max_bt_size_inter) and information about the difference between the maximum size and the minimum size of a binary divisible codec unit (log2_diff_ctu_max_bt_size_intra, log2_diff_ctu_max_bt_size_inter) may be signaled by an intra type and an inter type. Furthermore, information about the maximum size of a trifurcatingly divisible codec unit (log2_diff_ctu_max_tt_size_intra, log2_diff_ctu_max_tt_size_inter) and information about the difference between the maximum size and the minimum size of a trifurcatingly divisible codec unit (log2_diff_ctu_max_tt_size_intra, log2_diff_ctu_max_tt_size_inter) may be signaled through intra slices and inter slices through SPS.

[0655] In addition, information indicating whether the division mode is determined separately for the luminance block and the chrominance block (qtbtt_dual_tree_inter_flag) can be signaled through the SPS. When qtbtt_dual_tree_inter_fla indicates that the division mode is determined separately for the luminance block and the chrominance block, information about the difference between the maximum size and the minimum size of the chrominance codec unit that can be divided into two branches (log2_diff_ctu_max_bt_size_chroma_intra, log2_diff_ctu_max_bt_size_chroma_inter) can be signaled through the intra-frame slice and the inter-frame slice, through the SPS. In addition, information about the difference between the maximum size and the minimum size of the chrominance codec unit that can be divided into three branches (log2_diff_ctu_max_tt_size_chroma_intra, log2_diff_ctu_max_tt_size_chroma_inter) can be signaled through the intra-frame slice and the inter-frame slice, through the SPS.

[0656] Alternatively to the embodiment provided in FIG. 30 , in which information about the maximum size of a binary-dividable codec unit or a tri-dividable codec unit is signaled, information about the minimum size of a binary-dividable codec unit and information about the minimum size of a tri-dividable codec unit may be signaled.

[0657] In another embodiment, the signaling of the information about the size of the binary-dividable / trifurcably-dividable codec unit may not be performed through binary and trifurcably partitioning, but may be performed using common parameters.

[0658] Furthermore, additionally set values ​​for each slice may be updated and signaled at the slice header level.

[0659] The present invention has been specifically shown and described with reference to the embodiments of the present invention. To this end, it will be appreciated by those skilled in the art that various changes may be made to the form and details thereof without departing from the scope of the present disclosure. Therefore, the embodiments should be considered only in a descriptive sense and not for the purpose of limitation. The scope of the present invention is defined not by the detailed description of the present invention but by the appended claims, and all differences within the scope will be interpreted as included in the present disclosure.

[0660] At the same time, the above-mentioned embodiments of the present invention can be written as a program executable on a computer, and can be implemented in a general-purpose digital computer that executes the program by using a computer-readable recording medium. Examples of computer-readable storage media include magnetic storage media (such as ROM, floppy disks, hard disks, etc.), optical storage media (such as CD-ROM or DVD), etc.

Claims

1. A video decoding method, include: Identifying that the boundary of the first codec unit includes a lower boundary of the image; When the width of the first codec unit is greater than the maximum transform size and the height of the first codec unit is equal to the maximum transform size, determining that horizontal binary partitioning of the first codec unit is not allowed and vertical binary partitioning of the first codec unit is allowed; When horizontal binary division of the first codec unit is not allowed and vertical binary division of the first codec unit is allowed, determining a second codec unit by dividing the width of the first codec unit into halves in a vertical direction, and decoding the second codec unit; When the height of the first codec unit is greater than the maximum transform size and the width of the first codec unit is equal to the maximum transform size, determining that vertical binary partitioning of the first codec unit is not allowed and horizontal binary partitioning of the first codec unit is allowed; and When vertical binary division of the first codec unit is not allowed and horizontal binary division of the first codec unit is allowed, a third codec unit is determined by dividing the height of the first codec unit into halves in a horizontal direction and the third codec unit is decoded.

2. A video encoding method, include: Identifying that the boundary of the first codec unit includes a lower boundary of the image; When the width of the first codec unit is greater than the maximum transform size and the height of the first codec unit is equal to the maximum transform size, determining that horizontal binary partitioning of the first codec unit is not allowed and vertical binary partitioning of the first codec unit is allowed; When horizontal binary division of the first codec unit is not allowed and vertical binary division of the first codec unit is allowed, determining a second codec unit by dividing the width of the first codec unit into halves in a vertical direction, and encoding the second codec unit; When the height of the first codec unit is greater than the maximum transform size and the width of the first codec unit is equal to the maximum transform size, determining that a vertical binary split of the first codec unit is not allowed and a horizontal binary split of the first codec unit is allowed; When vertical binary division of the first codec unit is not allowed and horizontal binary division of the first codec unit is allowed, determining a third codec unit by dividing the height of the first codec unit into halves in a horizontal direction, and encoding the third codec unit; as well as A bit stream including the encoded data of the second codec unit or the encoded data of the third codec unit is generated.

3. A method for transmitting a bit stream generated by the video encoding method of claim 2.