IMAGE DECODING AND ENCODING METHODS AND BIT STREAM DATA TRANSMISSION METHODS

Abstract

This invention relates to a method for decoding and encoding images and a method for transmitting bitstream data. Specifically, the image decoding method may include the following steps: dividing the current encoded image block into multiple encoded image blocks such that the encoded image blocks divided from the current encoded image block are within the current frame when the current encoded image block is outside the boundaries of the current frame; parsing the first syntax element indicating whether the current encoded image block is divided into multiple sub-blocks when the current encoded image block satisfies a predefined condition; and determining the splitting mode of the current encoded image block based on the syntax element.

Description

Method for processing images and apparatus for the same

[0001] The present invention relates to a method for processing still images or video, and more specifically, to a method for determining the partitioning structure of a block on which encoding / decoding is performed and an apparatus supporting the same.

[0002] Compression encoding refers to a series of signal processing techniques used to transmit digitized information over communication lines or store it in a format suitable for storage media. Media such as video, images, and audio can be the subject of compression encoding; specifically, the technology of performing compression encoding on video is called video image compression.

[0003] Next-generation video content will be characterized by high spatial resolution, high frame rates, and high dimensionality of scene representation. Processing such content will result in a massive increase in memory storage, memory access rates, and processing power.

[0004] Therefore, it is necessary to design coding tools to process next-generation video content more efficiently.

[0005] The objective of the present invention is to propose a method for efficiently coordinating block partitioning when performing block partitioning with a QTBT structure.

[0006] The technical problems to be solved by the present invention are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which the present invention belongs from the description below.

[0007] One aspect of the present invention is a method for decoding an image, comprising: a step of dividing a current coding tree block into a plurality of coding blocks such that, when the current coding tree block extends beyond the boundary of the current picture, the coding blocks divided from the current coding tree block are included within the current picture; a step of parsing a first syntax element indicating whether the current coding block is divided into a plurality of sub-blocks when the current coding block satisfies a preset condition; and a step of parsing a second syntax element indicating whether the current coding block is divided using a quad-tree structure when the first syntax element indicates that the current coding block is divided. If the second syntax element indicates that the current coding block is not split using a quad tree structure, the method may include a step of parsing at least one of a third syntax element indicating whether the current coding block is split using a binary tree structure or a ternary tree structure, and a fourth syntax element indicating the splitting direction of the current coding block; and a step of determining a split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element.

[0008] Preferably, the step of dividing the current coding tree block into a plurality of coding blocks is performed by dividing the current coding tree block into a plurality of coding blocks using a binary tree structure or a binary tree structure until a valid region within the current coding tree block is reached, wherein the valid region may represent a region within the current coding tree block that falls within the boundary of the current picture.

[0009] Preferably, the step of dividing the current coding tree block into a plurality of coding blocks further includes the step of parsing a fifth syntax element indicating whether quad tree division is allowed when the current coding tree block goes beyond the boundary of the current partition, and when the fifth syntax element indicates that quad tree division is not allowed, the current coding tree block may be divided into a plurality of coding blocks using a binary tree structure or a binary tree structure.

[0010] Preferably, the fifth syntax element can be signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a header of a Network Abstract Layer unit.

[0011] Preferably, the step of dividing the current coding tree block into a plurality of coding blocks is performed by dividing the current coding tree block into a plurality of coding blocks using a quad tree structure until a valid region within the current coding tree block is reached, wherein the valid region may represent a region within the current coding tree block that falls within the boundary of the current picture.

[0012] Preferably, the step of dividing the current coding tree block into a plurality of coding blocks can be performed by dividing the current coding tree block into coding blocks using a quad tree structure when the width of the valid area is greater than the maximum transformation size or the height of the valid area is greater than the maximum transformation size.

[0013] Another aspect of the present invention is an apparatus for decoding an image, comprising: a coding tree block dividing unit that divides a current coding tree block into a plurality of coding blocks such that, when a current coding tree block goes beyond the boundary of a current picture, the coding blocks divided from the current coding tree block are included within the current picture; A syntax element parsing unit that, when the current coding block satisfies a preset condition, parses a first syntax element indicating whether the current coding block is divided into a plurality of sub-blocks; when the first syntax element indicates that the current coding block is divided, parses a second syntax element indicating whether the current coding block is divided using a quad-tree structure; and when the second syntax element indicates that the current coding block is not divided using a quad-tree structure, parses at least one of a third syntax element indicating whether the current coding block is divided using a binary-tree structure or a ternary-tree structure, and a fourth syntax element indicating the direction of division of the current coding block; and may include a split mode determining unit that determines the split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element and the fourth syntax element.

[0014] Preferably, the coding tree block splitting unit divides the current coding tree block into a plurality of coding blocks using a binary tree structure or a binary tree structure until a valid region within the current coding tree block is reached, wherein the valid region may represent a region within the current coding tree block that falls within the boundary of the current picture.

[0015] Preferably, the coding tree block splitting unit parses a fifth syntax element indicating whether quad tree splitting is allowed when the current coding tree block goes beyond the boundary of the current partition, and if the fifth syntax element indicates that quad tree splitting is not allowed, the current coding tree block can be split into a plurality of coding blocks using a binary tree structure or a binary tree structure.

[0016] Preferably, the fifth syntax element can be signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a header of a Network Abstract Layer unit.

[0017] Preferably, the coding tree block splitting unit divides the current coding tree block into a plurality of coding blocks using a quad tree structure until a valid region within the current coding tree block is reached, wherein the valid region may represent a region within the current coding tree block that falls within the boundary of the current picture.

[0018] Preferably, the coding tree block splitter can divide the current coding tree block into coding blocks using a quad tree structure when the width of the valid area is greater than the maximum transformation size or the height of the valid area is greater than the maximum transformation size.

[0019] According to an embodiment of the present invention, compression performance can be improved by effectively determining the QTBT partitioning structure and signaling related information.

[0020] In addition, according to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0021] The effects obtainable from the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art from the description below.

[0022] The accompanying drawings, which are included as part of the detailed description to aid in understanding the present invention, provide embodiments of the present invention and explain the technical features of the present invention together with the detailed description.

[0023] FIG. 1 shows a schematic block diagram of an encoding device in which video / image signal encoding is performed, as an embodiment to which the present invention is applied.

[0024] FIG. 2 shows a schematic block diagram of a decoding device in which video / image signal decoding is performed, as an embodiment to which the present invention is applied.

[0025] FIG. 3 is a diagram showing an example of a multitype tree structure as an embodiment to which the present invention can be applied.

[0026] FIG. 4 is a diagram illustrating a signaling mechanism for partition splitting information of a quadtree with nested multi-type tree structure, as an embodiment to which the present invention can be applied.

[0027] FIG. 5 is a diagram illustrating a method of dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure, as an embodiment to which the present invention can be applied.

[0028] FIG. 6 is a diagram illustrating a method for limiting ternary-tree partitioning as an embodiment to which the present invention can be applied.

[0029] FIG. 7 is a diagram illustrating a quad-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0030] FIG. 8 is a diagram illustrating a binary-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0031] FIG. 9 is a diagram illustrating a ternary-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0032] FIG. 10 is a diagram illustrating a block partitioning structure based on binary-tree, ternary-tree, and stripe tree structures as an embodiment to which the present invention is applied.

[0033] FIG. 11 is a drawing illustrating a block division structure of a square block as an embodiment to which the present invention is applied.

[0034] FIG. 12 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0035] FIG. 13 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0036] FIG. 14 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0037] FIG. 15 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0038] FIG. 16 is a drawing illustrating a block division structure of a square block as an embodiment to which the present invention is applied.

[0039] FIG. 17 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0040] FIG. 18 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0041] FIG. 19 is a diagram illustrating a case where a Coding Tree Unit (CTU) moves out of the boundary of an image as an embodiment to which the present invention is applied.

[0042] FIG. 20 is a diagram illustrating an asymmetric-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0043] FIGS. 21 to 23 are drawings illustrating examples of block division according to an embodiment to which the present invention can be applied.

[0044] FIG. 24 is a diagram illustrating a method for determining whether to split a block based on the maximum conversion block size without syntax signaling, as an embodiment to which the present invention is applied.

[0045] FIG. 25 is a flowchart illustrating a method for decoding a video signal according to an embodiment to which the present invention is applied.

[0046] FIG. 26 is a drawing illustrating a decoding device for a video signal according to an embodiment to which the present invention is applied.

[0047] FIG. 27 shows a video coding system to which the present invention is applied.

[0048] FIG. 28 shows a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0049] Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. The detailed description disclosed below, together with the accompanying drawings, is intended to describe exemplary embodiments of the present invention and is not intended to represent the only embodiment in which the present invention may be practiced. The following detailed description includes specific details to provide a complete understanding of the present invention. However, those skilled in the art will know that the present invention may be practiced without such specific details.

[0050] In some cases, to avoid obscuring the concept of the present invention, known structures and devices may be omitted or illustrated in the form of a block diagram focusing on the core functions of each structure and device.

[0051] In addition, the terms used in this invention have been selected to be as widely used as possible, but in specific cases, terms arbitrarily selected by the applicant are used for explanation. In such cases, the meaning is clearly stated in the detailed description of the relevant part, so it should be noted that the terms used in the description of this invention should not be interpreted simply by their names alone, but should be interpreted by grasping the meaning of the relevant terms.

[0052] Specific terms used in the following description are provided to aid in understanding the present invention, and the use of such specific terms may be modified in other forms without departing from the technical spirit of the present invention. For example, terms such as signal, data, sample, picture, frame, and block may be appropriately substituted and interpreted during each coding process.

[0053] In the present specification, the term "processing unit" refers to a unit in which encoding / decoding processing steps, such as prediction, transformation, and / or quantization, are performed. For convenience of explanation, the processing unit may also be referred to as a "processing block" or "block."

[0054] The processing unit can be interpreted to include units for the luminance (luma) component and units for the chroma component. For example, the processing unit may correspond to a Coding Tree Unit (CTU), Coding Unit (CU), Prediction Unit (PU), or Transform Unit (TU).

[0055] Additionally, the processing unit may be interpreted as a unit for the luminance (luma) component or a unit for the chroma component. For example, the processing unit may correspond to a Coding Tree Block (CTB), Coding Block (CB), Prediction Block (PU), or Transform Block (TB) for the luminance (luma) component. Alternatively, it may correspond to a Coding Tree Block (CTB), Coding Block (CB), Prediction Block (PU), or Transform Block (TB) for the chroma component. Furthermore, without being limited thereto, the processing unit may also be interpreted to include both the unit for the luminance (luma) component and the unit for the chroma component.

[0056] In addition, the processing unit is not necessarily limited to a square block and may be configured in the shape of a polygon with three or more vertices.

[0057] Additionally, in the present specification, pixels or pixels, etc. are collectively referred to as samples. Furthermore, using samples may mean using pixel values ​​or pixel values, etc.

[0058] FIG. 1 shows a schematic block diagram of an encoding device in which video / image signal encoding is performed, as an embodiment to which the present invention is applied.

[0059] Referring to FIG. 1, the encoding device (100) may be configured to include an image segmentation unit (110), a subtraction unit (115), a conversion unit (120), a quantization unit (130), an inverse quantization unit (140), an inverse conversion unit (150), an addition unit (155), a filtering unit (160), a memory (170), an inter prediction unit (180), an intra prediction unit (185), and an entropy encoding unit (190). The inter prediction unit (180) and the intra prediction unit (185) may be collectively referred to as the prediction unit. In other words, the prediction unit may include the inter prediction unit (180) and the intra prediction unit (185). The conversion unit (120), the quantization unit (130), the inverse quantization unit (140), and the inverse conversion unit (150) may be included in the residual processing unit. The residual processing unit may further include a subtraction unit (115). In one embodiment, the above-described image segmentation unit (110), subtraction unit (115), conversion unit (120), quantization unit (130), inverse quantization unit (140), inverse conversion unit (150), addition unit (155), filtering unit (160), inter prediction unit (180), intra prediction unit (185), and entropy encoding unit (190) may be configured by a single hardware component (e.g., an encoder or a processor). Additionally, the memory (170) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0060] The image segmentation unit (110) can divide an input image (or picture, frame) input to an encoding device (100) into one or more processing units. For example, the processing unit may be called a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBT (Quad-tree binary-tree) structure. For example, one coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure and / or a binary tree structure. In this case, for example, the quad-tree structure may be applied first and the binary tree structure may be applied later. Or the binary tree structure may be applied first. A coding procedure according to the present invention may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the term "coding procedure" may include procedures such as prediction, transformation, and restoration described below. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may each be divided or partitioned from the final coding unit described above. The prediction unit may be a unit for sample prediction, and the transformation unit may be a unit for deriving transformation coefficients and / or a unit for deriving a residual signal from transformation coefficients.

[0061] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0062] The encoding device (100) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (180) or an intra prediction unit (185) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (120). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoder (100) may be called a subtraction unit (115). The prediction unit performs a prediction for a block to be processed (hereinafter referred to as the current block) and can generate a predicted block containing prediction samples for the current block. The prediction unit can determine whether intra prediction is applied or inter prediction is applied at the current block or CU level. The prediction unit can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (190). The information regarding prediction can be encoded in the entropy encoding unit (190) and output in the form of a bitstream.

[0063] The intra prediction unit (185) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (185) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0064] The inter prediction unit (180) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different. The above temporal surrounding blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc., and the reference picture containing the above temporal surrounding blocks may be referred to as a collocated picture (colPic). For example, the inter prediction unit (180) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (180) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0065] The prediction signal generated through the above inter prediction unit (180) or the above intra prediction unit (185) can be used to generate a restoration signal or to generate a residual signal.

[0066] The transformation unit (120) can generate transform coefficients by applying a transformation technique to a residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. Additionally, the transformation process may be applied to a block of pixels of the same size in a square, or to a block of variable size that is not square.

[0067] The quantization unit (130) quantizes the transformation coefficients and transmits them to the entropy encoding unit (190), and the entropy encoding unit (190) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (130) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients. The entropy encoding unit (190) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (190) may encode information necessary for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to quantized transformation coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The bitstream may be transmitted through a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) that transmits the signal output from the entropy encoding unit (190) and / or a storage unit (not shown) that stores it may be configured as internal / external elements of the encoding device (100), or the transmission unit may be a component of the entropy encoding unit (190).

[0068] The quantized transformation coefficients output from the quantization unit (130) can be used to generate a prediction signal. For example, the quantized transformation coefficients can restore a residual signal by applying inverse quantization and inverse transformation through the inverse quantization unit (140) and inverse transformation unit (150) within the loop. The adder (155) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the restored residual signal to the prediction signal output from the inter-prediction unit (180) or intra-prediction unit (185). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (155) may be called a reconstructed unit or a reconstructed block generation unit. The generated restoration signal can be used for intra prediction of the next processing target block within the current picture, and can also be used for inter prediction of the next picture after filtering as described below.

[0069] The filtering unit (160) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (160) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (170), specifically in the DPB of memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (160) can generate various information regarding filtering and transmit it to the entropy encoding unit (190), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (190) and output in the form of a bitstream.

[0070] The modified restored picture transmitted to the memory (170) can be used as a reference picture in the inter-prediction unit (180). Through this, when inter-prediction is applied, the encoding device can avoid prediction mismatches between the encoding device (100) and the decoding device, and can also improve encoding efficiency.

[0071] The memory (170) DPB can store the modified restored picture to be used as a reference picture in the inter-prediction unit (180). The memory (170) can store motion information of blocks from which motion information is derived (or encoded) within the current picture and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (180) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (170) can store restoration samples of the blocks restored within the current picture and transmit them to the intra-prediction unit (185).

[0072] FIG. 2 shows a schematic block diagram of a decoding device in which video / image signal decoding is performed, as an embodiment to which the present invention is applied.

[0073] Referring to FIG. 2, the decoding device (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an adder (235), a filtering unit (240), a memory (250), an inter prediction unit (260), and an intra prediction unit (265). The inter prediction unit (260) and the intra prediction unit (265) may be collectively referred to as the prediction unit. That is, the prediction unit may include an inter prediction unit (180) and an intra prediction unit (185). The inverse quantization unit (220) and the inverse transformation unit (230) may be collectively referred to as the residual processing unit. That is, the residual processing unit may include an inverse quantization unit (220) and an inverse transformation unit (230). The above-described entropy decoding unit (210), inverse quantization unit (220), inverse transformation unit (230), addition unit (235), filtering unit (240), inter prediction unit (260), and intra prediction unit (265) may be configured by a single hardware component (e.g., a decoder or a processor) according to an embodiment. Additionally, the memory (170) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0074] When a bitstream containing video / image information is input, the decoding device (200) can restore the image in correspondence with the process in which the video / image information is processed in the encoding device of FIG. 1. For example, the decoding device (200) can perform decoding using a processing unit applied in the encoding device. Thus, the processing unit for decoding may be, for example, a coding unit, and the coding unit may be divided along a quad tree structure and / or a binary tree structure from a coding tree unit or a maximum coding unit. And, the restored image signal decoded and output through the decoding device (200) can be played back through a playback device.

[0075] The decoding device (200) can receive a signal output from the encoding device of FIG. 1 in the form of a bitstream, and the received signal can be decoded through the entropy decoding unit (210). For example, the entropy decoding unit (210) can parse the bitstream to derive information (e.g., video / image information) required for image restoration (or picture restoration). For example, the entropy decoding unit (210) can decode information within the bitstream based on coding methods such as exponential coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntactic element in a bitstream, determines a context model using information on the syntactic element to be decoded, decoding information of surrounding and target blocks, or information on symbols / bins decoded in the previous step, predicts the probability of bin occurrence according to the determined context model, and performs arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntactic element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using information on the decoded symbols / bins for the context model of the next symbol / bin. Information regarding prediction among the information decoded in the entropy decoding unit (2110) is provided to the prediction unit (inter prediction unit (260) and intra prediction unit (265)), and residual values, i.e., quantized transformation coefficients and related parameter information, from which entropy decoding is performed in the entropy decoding unit (210) can be input to the inverse quantization unit (220). Additionally, information regarding filtering among the information decoded in the entropy decoding unit (210) can be provided to the filtering unit (240).Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (200), or the receiving unit may be a component of the entropy decoding unit (210).

[0076] In the inverse quantization unit (220), the quantized transform coefficients can be inversely quantized to output transform coefficients. The inverse quantization unit (220) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit (220) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0077] In the inverse conversion unit (230), the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0078] The prediction unit performs a prediction for the current block and can generate a predicted block containing prediction samples for the current block. Based on information regarding the prediction output from the entropy decoding unit (210), the prediction unit can determine whether an intra prediction or an inter prediction is applied to the current block and can determine a specific intra / inter prediction mode.

[0079] The intra prediction unit (265) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (265) may determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0080] The inter prediction unit (260) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (260) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes, and information regarding the prediction may include information indicating the mode of inter-prediction for the current block.

[0081] The adder (235) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the inter prediction unit (260) or the intra prediction unit (265). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block.

[0082] The addition unit (235) may be called a restoration unit or a restoration block generation unit. The generated restoration signal may be used for intra-predicting the next block to be processed within the current picture, and may also be used for inter-predicting the next picture after filtering as described below.

[0083]

[0084] The filtering unit (240) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (240) can generate a modified restored picture by applying various filtering methods to the restored picture, and can transmit the modified restored picture to the memory (250), specifically to the DPB of the memory (250). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0085] The (modified) restored picture stored in the DPB of the memory (250) can be used as a reference picture in the inter-prediction unit (260). The memory (250) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (260) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (170) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (265).

[0086] In this specification, the embodiments described in the filtering unit (160), inter prediction unit (180), and intra prediction unit (185) of the encoding device (100) may be applied to the filtering unit (240), inter prediction unit (260), and intra prediction unit (265) of the decoding device (200) in the same or corresponding manner.

[0087]

[0088] Block Partitioning

[0089] The video / image coding method according to this document may be performed based on various detailed techniques, and each of these detailed techniques is outlined as follows. It is obvious to those skilled in the art that the techniques described below may be related to relevant procedures such as prediction, residual processing ((inverse)transform, (inverse)quantization, etc.), syntax element coding, filtering, and partitioning / segmentation in the video / image encoding / decoding procedures described above and / or below.

[0090] The block partitioning procedure according to the present document is performed in the image splitting unit (110) of the encoding device described above, and partitioning-related information can be processed (encoded) in the entropy encoding unit (190) and transmitted to the decoding device in the form of a bitstream. The entropy decoding unit (210) of the decoding device can derive the block partitioning structure of the current picture based on the partitioning-related information obtained from the bitstream, and perform a series of procedures for image decoding (e.g., prediction, residual processing, block restoration, in-loop filtering, etc.) based thereon.

[0091] Partitioning of picture into CTUs

[0092] Pictures can be divided into a sequence of Coding Tree Units (CTUs). A CTU may correspond to a Coding Tree Block (CTB). Alternatively, a CTU may include a Coding Tree Block of Luma Samples and two Coding Tree Blocks of corresponding Chroma Samples. In other words, for a picture containing three sample arrays, a CTU may include an NxN block of Luma Samples and two corresponding blocks of Chroma Samples.

[0093] The maximum allowable size of a CTU for coding and prediction, etc., may differ from the maximum allowable size of a CTU for transformation. For example, the maximum allowable size of a luma block within a CTU may be 128x128.

[0094] Partitioning of the CTUs using a tree structure

[0095] A CTU can be partitioned into CUs based on a quad-tree (QT) structure. The quad-tree structure can be referred to as a quaternary tree structure. This is intended to reflect various local characteristics. Meanwhile, in this document, a CTU can be partitioned based on a multitype tree structure partitioning that includes not only quad-trees but also binary trees (BT) and ternary trees (TT). Hereinafter, the term QTBT structure may include quad-tree and binary tree-based partitioning structures, and QTBTTT may include quad-tree, binary tree, and ternary tree-based partitioning structures. Alternatively, the QTBT structure may include quad-tree, binary tree, and ternary tree-based partitioning structures. In a coding tree structure, CUs can have a square or rectangular shape. A CTU can first be partitioned into a quad-tree structure. Subsequently, the leaf nodes of the quad-tree structure can be further partitioned by a multitype tree structure.

[0096] FIG. 3 is a diagram showing an example of a multitype tree structure as an embodiment to which the present invention can be applied.

[0097] In one embodiment of the present invention, the multitype tree structure may include four splitting types as illustrated in FIG. 3. The four splitting types may include vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), and horizontal ternary splitting (SPLIT_TT_HOR). The leaf nodes of the multitype tree structure may be called CUs. These CUs may be used for prediction and transformation procedures. In this document, CUs, PUs, and TUs may generally have the same block size. However, if the maximum supported transform length is smaller than the width or height of the color component of the CU, the CUs and TUs may have different block sizes.

[0098] FIG. 4 is a diagram illustrating a signaling mechanism for partition splitting information of a quadtree with nested multi-type tree structure, as an embodiment to which the present invention can be applied.

[0099] Here, the CTU is treated as the root of the quadtree and is initially partitioned into a quadtree structure. Each quadtree leaf node can subsequently be further partitioned into a multitype tree structure. In the multitype tree structure, a first flag (e.g., mtt_split_cu_flag) is signaled to indicate whether the node is further partitioned. If the node is further partitioned, a second flag (e.g., mtt_split_cu_vertical_flag) may be signaled to indicate the splitting direction. Subsequently, a third flag (e.g., mtt_split_cu_binary_flag) may be signaled to indicate whether the splitting type is binary or binary. For example, based on the above mtt_split_cu_vertical_flag and the above mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1 below.

[0100]

[0101] FIG. 5 is a diagram illustrating a method of dividing a CTU into multiple CUs based on a quadtree and nested multi-type tree structure, as an embodiment to which the present invention can be applied.

[0102] Here, bold block edges represent quadtree partitioning, and the remaining edges represent multitype tree partitioning. Quadtree partitioning involving a multitype tree can provide a content-adapted coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU may include a coding block of luminance samples and two coding blocks of corresponding chroma samples. The size of a CU may be as large as a CTU, or it may be 4x4 in luminance sample units. For example, in the case of a 4:2:0 color format (or chroma format), the maximum chroma CB size may be 64x64 and the minimum chroma CB size may be 2x2.

[0103] For example, in this document, the maximum allowable Luma TB size may be 64x64 and the maximum allowable Chroma TB size may be 32x32. If the width or height of a CB partitioned according to the above tree structure is greater than the maximum conversion width or height, the CB may be automatically (or implicitly) partitioned until the horizontal and vertical TB size limits are satisfied.

[0104] Meanwhile, for a quadtree coding tree scheme involving a multitype tree, the following parameters can be defined and identified as SPS syntax elements.

[0105] - CTU size: the root node size of a quaternary tree

[0106] - MinQTSize: the minimum allowed quaternary tree leaf node size

[0107] - MaxBtSize: the maximum allowed binary tree root node size

[0108] - MaxTtSize: the maximum allowed ternary tree root node size

[0109] - MaxMttDepth: the maximum allowed hierarchy depth of multi-type tree splitting from a quadtree leaf

[0110] - MinBtSize: the minimum allowed binary tree leaf node size

[0111] - MinTtSize: the minimum allowed ternary tree leaf node size

[0112] As an example of a quadtree coding tree structure involving a multitype tree, the CTU size can be set to 64x64 blocks of 128x128 luminance samples and two corresponding chroma samples (in the 4:2:0 chroma format). In this case, MinOTSize can be set to 16x16, MaxBtSize to 128x128, MaxTtSize to 64x64, MinBtSize and MinTtSize (for both width and height) to 4x4, and MaxMttDepth to 4. Quadtree partitioning can be applied to the CTU to create quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. Quadtree leaf nodes can have sizes ranging from 16x16 (i.e., the MinOTSize) to 128x128 (i.e., the CTU size). If the leaf QT node is 128x128, it may not be further split into a binary tree / binary tree. This is because even if it were split in this case, it would exceed MaxBtsize and MaxTtsize (i.e., 64x64). Otherwise, the leaf QT node may be further split into a multitype tree. Therefore, the leaf QT node is the root node of the multitype tree, and the leaf QT node can have a multitype tree depth (mttDepth) value of 0. If the multitype tree depth reaches MaxMttdepth (e.g., 4), further splitting may not be considered. If the width of the multitype tree node is equal to MinBtSize and is less than or equal to 2xMinTtSize, further horizontal splitting may not be considered. If the height of a multitype tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, no further vertical splitting may be considered.

[0113] FIG. 6 is a diagram illustrating a method for limiting ternary-tree partitioning as an embodiment to which the present invention can be applied.

[0114] Referring to FIG. 6, in order to allow a 64x64 luminance block and a 32x32 chroma pipeline design in a hardware decoder, TT splitting may be limited in certain cases. For example, if the width or height of the luminance coding block is greater than a preset value (e.g., 32, 64), TT splitting may be limited as shown in FIG. 6.

[0115] In this document, the coding tree scheme may support luminance and chroma blocks having separate block tree structures. For P and B slices, luminance and chroma CTBs within a single CTU may be restricted to having the same coding tree structure. However, for I slices, luminance and chroma blocks may have separate block tree structures. If the separate block tree mode is applied, luminance CTBs may be divided into CUs based on a specific coding tree structure, and chroma CTBs may be divided into chroma CUs based on a different coding tree structure. This may mean that CUs within an I slice may consist of coding blocks of luminance components or coding blocks of two chroma components, and CUs in a P or B slice may consist of blocks of three color components.

[0116] In the aforementioned “Partitioning of the CTUs using a tree structure,” a quadtree coding tree structure involving a multitype tree was described, but the structure in which the CU is partitioned is not limited to this. For example, the BT structure and the TT structure can be interpreted as concepts included in the Multiple Partitioning Tree (MPT) structure, and the CU can be interpreted as being partitioned through the QT structure and the MPT structure. In an example where the CU is partitioned through the QT structure and the MPT structure, the partitioning structure can be determined by signaling a syntax element (e.g., MPT_split_type) containing information regarding how many blocks the leaf node of the QT structure is partitioned into, and a syntax element (e.g., MPT_split_mode) containing information regarding whether the leaf node of the QT structure is partitioned vertically or horizontally.

[0117] In another example, the CU may be divided in a way different from the QT structure, BT structure, or TT structure. That is, unlike when a lower-depth CU is divided into 1 / 4 the size of an upper-depth CU according to the QT structure, or a lower-depth CU is divided into 1 / 2 the size of an upper-depth CU according to the BT structure, or a lower-depth CU is divided into 1 / 4 or 1 / 2 the size of an upper-depth CU according to the TT structure, the lower-depth CU may, in some cases, be divided into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of an upper-depth CU, and the method of dividing the CU is not limited thereto.

[0118] If a portion of a tree node block exceeds the bottom or right picture boundary, the tree node block may be restricted so that all samples of all coded CUs are located within the said picture boundaries. In this case, for example, the following partitioning rule may be applied.

[0119] - If a portion of a tree node block exceeds both the bottom and the right picture boundaries,

[0120] - If the block is a QT node and the size of the block is larger than the minimum QT size, the block is forced to be split with QT split mode.

[0121] -Otherwise, the block is forced to be split with SPLIT_BT_HOR mode

[0122] -Otherwise if a portion of a tree node block exceeds the bottom picture boundaries,

[0123] - If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forced to be split with QT split mode.

[0124] - Otherwise, if the block is a QT node, and the size of the block is larger than the minimum QT size and the size of the block is smaller than or equal to the maximum BT size, the block is forced to be split with QT split mode or SPLIT_BT_HOR mode.

[0125] - Otherwise (the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is forced to be split with SPLIT_BT_HOR mode.

[0126] - Otherwise if a portion of a tree node block exceeds the right picture boundaries,

[0127] - If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forced to be split with QT split mode.

[0128] -Otherwise, if the block is a QT node, and the size of the block is larger than the minimum QT size and the size of the block is smaller than or equal to the maximum BT size, the block is forced to be split with QT split mode or SPLIT_BT_VER mode.

[0129] -Otherwise (the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is forced to be split with SPLIT_BT_VER mode.

[0130]

[0131] Example 1

[0132] FIG. 7 is a diagram illustrating a quad-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0133] Referring to FIG. 7, in one embodiment of the present invention, a block may be partitioned based on a quad-tree (QT). Additionally, a sub-block partitioned by a QT structure may be recursively further partitioned by a QT structure. A terminal block that is no longer partitioned by a QT structure (which may be referred to as a leaf block or leaf node block) may be partitioned by at least one binary tree (BT), ternary tree (TT), or asymmetric tree (AT).

[0134] For example, block A can be divided into four sub-blocks (A0, A1, A2, A3) using a QT structure. And, sub-block A1 can again be divided into four sub-blocks (B0, B1, B2, B3) using a QT structure.

[0135] FIG. 8 is a diagram illustrating a binary-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0136] Referring to Fig. 8, the BT can have two forms of division: a horizontal BT (e.g., 2NxN, 2NxN) and a vertical BT (e.g., Nx2N, Nx2N).

[0137] For example, a block B3 that is no longer divided into a QT structure (i.e., a leaf node block) can be divided into vertical BT(C0, C1) or horizontal BT(D0, D1). When divided vertically, each sub-block, such as block C0, can be recursively divided into horizontal BT(E0, E1) or vertical BT(F0, F1).

[0138] FIG. 9 is a diagram illustrating a ternary-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0139] Referring to FIG. 9, TT can have two forms of division: horizontal TT (e.g., 2Nx1 / 2N, 2NxN, 2Nx1 / 2N) and vertical TT (e.g., 1 / 2Nx2N, Nx2N, 1 / 2Nx2N).

[0140] For example, block B3, which is no longer divided by QT, can be divided into vertical TT (C0, C1, C2) or horizontal TT (D0, D1, D2). As shown in FIG. 11, block C1 can be recursively divided into horizontal TT (E0, E1, E2) or vertical TT (F0, F1, F2).

[0141] As another example, AT can have four types of partitioning structures: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-up AT (1 / 2Nx2N, 3 / 2Nx2N), and vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N). Each BT, TT, AT can be recursively further partitioned using BT, TT, AT.

[0142] FIG. 10 is a diagram illustrating a block partitioning structure based on binary-tree, ternary-tree, and stripe tree structures as an embodiment to which the present invention is applied.

[0143] Referring to FIG. 10, the processing block can be divided using BT, TT, and stripe tree (ST) partitioning structures together. For example, a sub-block divided by a BT structure can be divided by a TT structure. Or, a sub-block divided by a TT structure can be divided by a BT structure.

[0144] Additionally, in one embodiment, the processing block may be divided into an ST structure that is divided into four parts in the horizontal or vertical direction. Furthermore, BT, TT, and ST divisions can be used together for division. For example, a sub-block divided by BT can be divided by TT or ST. Additionally, a sub-block divided by TT can be divided by BT or ST. Additionally, a sub-block divided by ST can be divided by BT or TT.

[0145] In Fig. 10, the thin solid line represents the first division, and the thick dotted line represents the second division performed in the sub-block created in the first division.

[0146] Table 2 below shows the syntax structure of a coding quadtree, and Table 3 shows the syntax for a coding tree.

[0147]

[0148] Refer to Table 2 to describe the decoding process for determining the quad tree partition structure. The coding quad tree syntax (or function) can be called with the top-left coordinates (x0, y0) of the current block, the size of the current block (log2CbSize), and the depth of the current quad tree partition as input.

[0149] If the current block does not exceed the width or height of the current picture and is larger than the minimum size coding block, the decoder parses the split_qt_flag syntax element. The split_qt_flag syntax element indicates whether the current block is divided into 4 quadtrees. For example, a split_qt_flag value of 0 indicates no division, while a split_qt_flag value of 1 indicates that the block is divided into 4 quadtrees with half the width and half the height. x0 and y0 represent the top-left position of the luminance image.

[0150] If the parsing result determines that the current block is to be divided into a quadtree structure, the decoder calls the coding quadtree syntax (or function) again for the four sub-blocks divided from the current block.

[0151] If the parsing result determines that the current block is not to be partitioned into a quad tree structure, the decoder calls the coding tree syntax (or function) on the current block to determine the subsequent partitioning structure (i.e., a multi-type tree structure).

[0152]

[0153] Refer to Table 3 to describe the decoding process for determining additional partition structures in a quad tree leaf node block. A coding tree syntax (or function) can be called with the top-left coordinates (x0, y0) of the current block, the width of the current block (log2CbSizeW), and the width of the current block (log2CbSizeH) as input.

[0154] If the current block does not exceed the width or height of the current picture and the current block is larger than the minimum size MTT block, the decoder parses the split_fur_flag syntax element. The split_fur_flag syntax indicates whether the current block is further split. For example, a split_fur_flag value of 0 means that it is not further split, and a split_fur_flag value of 1 means that the block is split.

[0155] If the parsing result confirms that the current block is further split, the decoder parses the split_bt_flag and split_dir syntax elements. The split_bt_flag syntax indicates whether the block is split into a BT structure or a TT structure. For example, if the split_bt_flag value is 1, it means that the block is split by BT, and if the split_bt_flag value is 0, it means that the block is split by TT.

[0156] The split type of the current block (SplitType) can be determined based on the split_fur_flag syntax and split_bt_flag syntax as shown in Table 4 below.

[0157]

[0158] In addition, in Table 3, the split_dir syntax element indicates the direction of splitting. For example, if the split_dir value is 0, it indicates that the block is split horizontally, and if the split_dir value is 1, it indicates that the block is split vertically.

[0159] The final block split mode (SplitMode) split from the current block MTT can be derived as shown in Table 5 below.

[0160]

[0161] The decoder calls the coding tree syntax (or function) again for the sub-blocks divided according to the finally determined partitioning structure.

[0162]

[0163] Example 2

[0164] In one embodiment of the present invention, a method for allowing BT, TT, or ST partitioning based on the shape (or size) of a block is proposed. In the above-described embodiment 1, partitioning is allowed regardless of the shape of the block if it does not reach MaxDepth, which represents the maximum partitioning depth, MinBTSize, which represents the minimum size of a block partitioned into a binary tree structure, or MinTTSize, which represents the minimum size of a block partitioned into a binary tree structure. However, in this embodiment, the encoder / decoder may adaptively allow block partitioning based on the width or height of the block (i.e., the shape of the block).

[0165] FIG. 11 is a drawing illustrating a block division structure of a square block as an embodiment to which the present invention is applied.

[0166] Referring to FIG. 11, it is assumed that the current block is a 2Nx2N block. In one embodiment of the present invention, a square block with equal width and height can be divided horizontally (BT), TT, and ST, as shown in FIG. 11, and can be divided vertically (BT, TT, and ST). That is, in an embodiment of the present invention, the square block may not be subject to block division restrictions based on the block shape.

[0167] FIG. 12 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0168] Referring to FIG. 12, it is assumed that the current block is a 2NxN block. In one embodiment of the present invention, for a non-square block where the width is twice the height, only vertical BT, TT division and horizontal BT division may be allowed as shown in FIG. 12.

[0169] In other words, if the current block is a non-square block where the width is greater than the height, ST division in the horizontal and vertical directions may not be allowed. Also, if the current block is a non-square block where the width is greater than the height, TT division in the horizontal direction may not be allowed.

[0170] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0171] FIG. 13 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0172] Referring to FIG. 13, it is assumed that the current block is an Nx2N block. In one embodiment of the present invention, for a non-square block where the height is twice the width, only horizontal BT, TT division and vertical BT division may be allowed as shown in FIG. 13.

[0173] In other words, if the current block is a non-square block where the height is greater than the width, ST division in the horizontal and vertical directions may not be allowed. Also, if the current block is a non-square block where the height is greater than the width, TT division in the vertical direction may not be allowed.

[0174] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0175] FIG. 14 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0176] Referring to FIG. 14, it is assumed that the current block is a 2Nx1 / 2N block. In one embodiment of the present invention, for a non-square block where the width is four times greater than the height, only vertical BT division may be allowed as shown in FIG. 14.

[0177] In other words, if the current block is a block with a width that is four times greater than its height, vertical TT and ST divisions may not be allowed. Additionally, if the current block is a block with a width that is four times greater than its height, all divisions in the horizontal direction may not be allowed.

[0178] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0179] FIG. 15 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0180] Referring to FIG. 15, it is assumed that the current block is a 1 / 2Nx2N block. In one embodiment of the present invention, a non-square block with a height four times greater than its width may only be allowed to undergo horizontal BT division as shown in FIG. 15.

[0181] In other words, if the current block is a block with a height that is four times greater than its width, horizontal TT and ST divisions may not be allowed. Also, if the current block is a block with a height that is four times greater than its width, all divisions in the vertical direction may not be allowed.

[0182] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0183] In one embodiment, the method of restricting specific divisions according to the shape or size of the block described above may be applied only when the boundary of the CTU (or CTB, maximum size CU) extends beyond the boundary of the image (picture or slice).

[0184]

[0185] Example 3

[0186] The present invention proposes a method for allowing BT, TT, or ST division based on the shape of a block. According to Example 2 described above, an encoder / decoder can adaptively allow block division based on the shape of the width and height of the block. Example 2 described above is a method that allows various block divisions for relatively large blocks, but does not allow specific divisions based on the block shape for relatively small blocks. On the other hand, the embodiment of the present invention proposes a method that allows more fine block division for relatively small blocks.

[0187] FIG. 16 is a drawing illustrating a block division structure of a square block as an embodiment to which the present invention is applied.

[0188] Referring to FIG. 16, it is assumed that the current block is a 2Nx2N block. In one embodiment of the present invention, a square block with equal width and height may only be allowed to have horizontal and vertical BT divisions, as shown in FIG. 16. That is, a square block with equal width and height may not be allowed to have TT and ST divisions.

[0189] In addition, in one embodiment of the present invention, possible partitioning for 2NxN blocks and Nx2N blocks may be allowed for the same partitioning structure as in Example 2 described above (see FIG. 12 and FIG. 13).

[0190] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0191] FIG. 17 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0192] Referring to FIG. 17, it is assumed that the current block is a 2Nx1 / 2N block. In one embodiment of the present invention, for a non-square block where the width is four times greater than the height, only vertical BT, TT, and ST divisions may be allowed as shown in FIG. 17.

[0193] In other words, if the current block is four times wider than its height, only vertical division may be allowed. Also, if the current block is four times wider than its height, no horizontal division may be allowed.

[0194] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0195] FIG. 18 is a drawing illustrating a block division structure of a non-square block as an embodiment to which the present invention is applied.

[0196] Referring to FIG. 18, it is assumed that the current block is a 1 / 2Nx2N block. In one embodiment of the present invention, for a non-square block where the height is four times greater than the width, only horizontal BT, TT, and ST divisions may be allowed as shown in FIG. 15.

[0197] In other words, if the current block is at least four times taller than its width, only horizontal division may be allowed. Also, if the current block is at least four times taller than its width, no vertical division may be allowed.

[0198] According to an embodiment of the present invention, by not allowing an inefficient partitioning structure by considering the ratio of width to height, the number of bits required for partitioning information signaling can be reduced.

[0199] In one embodiment, the method of restricting specific divisions according to the shape or size of the block described above may be applied only when the boundary of the CTU (or CTB, maximum size CU) extends beyond the boundary of the image (picture or slice).

[0200]

[0201] Example 4

[0202] In Examples 2 and 3 above, a method for determining a block partitioning structure that can be partitioned according to the shape of a block was described. In one embodiment of the present invention, the syntax for determining a block partitioning structure according to the block shape described above can be transmitted in a high-level syntax.

[0203] For example, the syntax for determining the block partitioning structure according to the above block shape can be signaled from the encoder to the decoder through the Sequence Parameter Set, Picture Parameter Set, Tile Group Header, or the header of the Network Abstract Layer unit.

[0204] In one embodiment, the block partitioning structure determination syntax can be transmitted in a sequence parameter set (SPS) as shown in Table 6 below.

[0205]

[0206] Referring to Table 6, log2_size_bt_allow_ratio is a syntax (or syntax element) that indicates the allowable range of the ratio (or difference, ratio of difference) between the width and height of a block for BT splitting. For example, if its value is 1, the encoder / decoder may allow splitting up to twice the length of one side of the block (i.e., width or height) compared to the other side, and may not allow block splitting if it is greater than that. Also, log2_size_bt_allow_ratio and log2_size_st_allow_ratio are syntax (or syntax elements) that indicate the allowable range of the ratio (or difference, ratio of difference) between the width and height of a block for TT and ST splitting, respectively.

[0207] Alternatively, in one embodiment, the block partitioning structure determination syntax may be transmitted in a sequence parameter set (SPS) as shown in Table 7 below.

[0208]

[0209] Referring to Table 7, log2_size_bt_disallow_ratio is a syntax (or syntax element) that indicates the allowable range of the ratio (or difference, ratio of difference) between the width and height of a block for BT splitting. For example, if its value is 1, the encoder / decoder may not allow the splitting of a block when the length of one side of the block becomes twice that of the other side. Also, log2_size_bt_disallow_ratio and log2_size_st_disallow_ratio are syntax (or syntax elements) that indicate the allowable range of the ratio (or difference, ratio of difference) between the width and height of a block for TT and ST splitting, respectively.

[0210] The syntax of Tables 6 and 7 can be signaled from the encoder to the decoder through the Sequence Parameter Set, Picture Parameter Set, Tile Group Header, or the header of the Network Abstract Layer unit.

[0211] According to the examples in Tables 6 and 7, when the width and height are in a constant ratio, block division is not allowed, thereby efficiently determining the division structure and reducing the signaling bits required for determining the division structure.

[0212] In one embodiment, the method of restricting specific divisions according to the shape or size of the block described above may be applied only when the boundary of the CTU (or CTB, maximum size CU) extends beyond the boundary of the image (picture or slice).

[0213]

[0214] Example 5

[0215] As described above, the QT structure can be used when the width and height of the blocks are equal. In one embodiment of the present invention, when a block spans across a picture boundary region, that is, when the left or lower boundary of the current CTU (or, maximum coding unit, CTB) exceeds the current picture, the encoder / decoder can divide the current CTU into a block of the maximum size that can be divided into a BT or TT (i.e., a multi-type tree (MTT)) without additional syntax signaling.

[0216] In conventional image compression technology, when the boundary of a CTU (or CTB, maximum size CU) extends beyond the boundary of a picture (or slice), the decoder can perform QT ​​segmentation without parsing the segmentation information signaled from the encoder (i.e., implicitly) until it extends beyond the boundary of the picture.

[0217] For example, if the current CTU is 64x64 and the picture (or slice) contains only a 64x32 area, the decoder can QT-divide the current CTU into four 32x32 CUs without parsing the division information. In such a case, the 64x32 blocks are implicitly divided into two 32x32 blocks, after which parsing of the division information is performed. In the present invention, implicit division refers to a method of performing block division according to specific conditions without signaling the division information, and may be referred to as automatic division, default division, etc.

[0218] In an embodiment of the present invention, a method is proposed to minimize CTU segmentation that extends beyond the picture boundary by not allowing QT segmentation when the boundary of the CTU (or CTB, maximum size CU) extends beyond the picture boundary. For example, if the encoder / decoder currently has a 64x64 CTU and the picture (or slice) contains only a 64x32 region, it may perform segmentation other than QT (e.g., BT, TT, AT) by not allowing QT segmentation. For example, BT segmentation may be performed instead of QT segmentation, in which case the 64x32 size block adjacent to the picture (or slice) boundary may be segmented starting from the 64x32 size block through parsing of segmentation information, rather than from two 32x32 size blocks. According to an embodiment of the present invention, by adjusting QT segmentation according to the complexity of the image, an effective segmentation structure can be determined with relatively little segmentation information.

[0219] In one embodiment, when the boundary of a CTU (or CTB, maximum size CU) extends beyond the boundary of a picture (or slice), the encoder may transmit a syntax indicating whether to allow QT splitting to the decoder. The syntax indicating whether to allow QT splitting may be transmitted, for example, at the level of a sequence, picture, or slice (or tile group). Table 8 below illustrates a QT splitting restriction syntax signaled at a higher level.

[0220]

[0221] Referring to Table 8, pic_boundary_qt_disable_flag represents a syntax (or syntax element) indicating whether to allow segmentation by QT when the block size of the image boundary region is not equal to the maximum block size. As an example, if the value of pic_boundary_qt_disable_flag is 1, QT segmentation may not be allowed in the image boundary region. The syntax may be signaled from the encoder to the decoder through the Picture Parameter Set, Tile Group Header, or the header of the Network Abstract Layer unit in addition to the Sequence Parameter Set.

[0222] FIG. 19 is a diagram illustrating a case where a Coding Tree Unit (CTU) moves out of the boundary of an image as an embodiment to which the present invention is applied.

[0223] Referring to FIG. 19, the current picture can be divided into CTU units. In this case, the height of the current picture may not be a multiple of the CTU height, and in this case, as shown in FIG. 19, the CTUs located at the bottom of the current picture may extend beyond the boundaries of the current picture.

[0224] That is, a possible block splitting shape is determined according to the block shape determined by the embodiments 2 to 4 described above, and the splitting can be performed implicitly from the largest block that can split the corresponding block. At this time, information for the splitting may not be included in the syntax. After being implicitly split into the largest block that can be split, the encoder can signal information about the optimal block splitting structure.

[0225] In addition, in another embodiment, when the boundary of the CTU (or CTB, maximum size CU) extends beyond the boundary of the picture (or slice), the encoder and decoder can determine whether to allow QT splitting according to specific conditions. If QT splitting is not allowed, the encoder / decoder can split the CTU using a splitting other than QT (e.g., BT, TT, AT).

[0226] For example, if the boundary of the CTU extends beyond the boundary of the image (picture or slice), the decoder may determine whether to allow QT splitting based on the overlapping area (or overlapping area) between the CTU and the image. For example, the decoder may not allow QT splitting if the overlapping area is less than or equal to 1 / 2 of the CTU size, and may allow QT splitting if the overlapping area exceeds 1 / 2.

[0227] At this time, prior to the step of determining whether to allow QT segmentation based on the overlapping area, a step of verifying (or determining) the overlapping area may be performed. The overlapping area may be calculated using the area ratio within the CTU, the width or height ratio relative to the CTU, the area / width / height ratio with the virtual area, etc. In the present invention, the remaining area within the CTU excluding the area where the CTU and the image overlap (i.e., the area extending beyond the boundaries of the image (picture or slice) within the CTU) may be referred to as the virtual area. Subsequently, if the overlapping area (or area, ratio) is smaller than or equal to a predetermined area (or area, ratio), QT segmentation may not be allowed, and if it is larger, QT segmentation may be allowed. Additionally, for example, the allowance of QT segmentation may be determined by comparing the ratio of the overlapping area with a specific threshold.

[0228] Additionally, for example, if the boundary of the CTU extends beyond the boundary of the image (picture or slice), the decoder may determine whether to allow QT partitioning or an implicit partitioning structure based on the size of the transform (or transform kernel, transform matrix). For example, the decoder may not allow QT partitioning if there is a transform available in the overlapping region. Alternatively, the decoder may determine whether to allow QT partitioning based on the overlapping region as in the embodiment described above, and determine the partitioning structure according to the size of the transform available in the overlapping region. For example, the decoder may partition the overlapping region in units of the maximum size of the available transform.

[0229] In one embodiment, the encoder / decoder can perform QT ​​partitioning on the current CTU without syntax signaling when the overlapping region of the current CTU is larger than the maximum transform size. For example, the encoder / decoder can perform QT ​​partitioning on the current CTU without syntax signaling when the width of the overlapping region, i.e., the region within the current picture excluding the virtual region within the CTU, is larger than the maximum transform size (or the length of one side of the maximum transform), or when the height of the overlapping region is larger than the maximum transform size.

[0230] Additionally, in another embodiment, if the boundary of a CTU (or CTB, maximum size CU) extends beyond the boundary of a picture (or slice), QT splitting may be configured not to be allowed at all. If QT splitting is not allowed, the CTU may be split using a splitting method other than QT (e.g., BT, TT, AT). In this case, the splitting of blocks extending beyond the picture boundary can be minimized while reducing syntax signaling data.

[0231] According to an embodiment of the present invention, when the boundary of a CTU (or CTB, maximum size CU) extends beyond the boundary of an image (picture or slice), the segmentation of the image within the CTU can be minimized, and the size of the initial segmentation block of the image within the CTU can be effectively determined. As image resolutions become larger and more diverse, the CTU (or CTB, maximum size CU), which is the basic unit for segmenting images, is also being discussed in larger sizes than existing HEVC, such as 128x128 and 256x256 blocks.

[0232] Therefore, cases where CTU boundaries extend beyond the image boundaries may occur frequently. If QT segmentation is allowed in all cases, as in conventional image compression techniques, the image boundaries may be segmented into unnecessarily small sizes. Consequently, as encoding information is signaled in units of these unnecessarily segmented blocks, compression efficiency at the image boundaries may be significantly degraded. Table 9 below shows typical image sizes.

[0233]

[0234] In the embodiments of the present invention, the case where the CTU size is 64x64 or 128x128 is described primarily, but the present invention is not limited thereto and can be applied in the same way to cases where the size is 256x256 or 512x512. In the case of 8k video, it may have a resolution of 7680x4320, and in this case, all CTUs belonging to the lowest CTU line may extend beyond the boundaries of the video. In this case, the embodiments described above may be applied. Similarly, in the case of 4k video having a resolution of 4096x2160, UHD video having a resolution of 3840x2160, FHD video having a resolution of 1920x1080, etc., all CTUs belonging to the lowest CTU line may extend beyond the boundaries of the video, and the embodiments described above may be applied.

[0235]

[0236] Example 6

[0237] FIG. 20 is a diagram illustrating an asymmetric-tree-based block partitioning structure as an embodiment to which the present invention is applied.

[0238] Referring to FIG. 20, the AT can have four types of divisions: horizontal-up AT (2Nx1 / 2N, 2Nx3 / 2N), horizontal-down AT (2Nx3 / 2N, 2Nx1 / 2N), vertical-left AT (1 / 2Nx2N, 3 / 2Nx2N), and vertical-right AT (3 / 2Nx2N, 1 / 2Nx2N).

[0239] For example, block B3, which is no longer divided by QT, can be divided into vertical AT (C0, C1) or horizontal AT (D0, D1). Each sub-block, such as block C1, can be recursively further divided into forms such as horizontal AT (E0, E1) or vertical TT (F0, F1).

[0240] FIGS. 21 to 23 are drawings illustrating examples of block division according to an embodiment to which the present invention can be applied.

[0241] Referring to FIGS. 21 to 23, BT, TT, and AT divisions can be used together for division. For example, a sub-block divided by BT can be divided by TT or AT. Additionally, a sub-block divided by TT can be divided by BT or AT. A sub-block divided by AT can be divided by BT or TT.

[0242] In FIGS. 21 to 23, the divisions represented by solid lines indicate the first division, and the divisions represented by dotted lines indicate the second division of the sub-blocks created by the first division. For example, FIGS. 21 (1) to (4) illustrate that after the horizontal BT division, each sub-block is divided into vertical BT, vertical TT, vertical left AT, and vertical right AT. FIGS. 21 (5) to (8) illustrates that after the horizontal TT division, each sub-block is divided into vertical BT, vertical TT, vertical left AT, and vertical right AT.

[0243] FIGS. 21 (9) to (12) illustrate that after the horizontal upper AT division, each sub-block is divided into a vertical BT, a vertical TT, a vertical left AT, and a vertical right AT. FIGS. 22 (13) to (16) illustrates that after the horizontal lower AT division, each sub-block is divided into a vertical BT, a vertical TT, a vertical left AT, and a vertical right AT.

[0244] FIGS. 22 (17) to (20) illustrate that after the vertical BT division, each sub-block is divided into horizontal BT, horizontal TT, horizontal upper AT, and horizontal lower AT. FIGS. 22 (21) to (24) illustrates that after the vertical TT division, each sub-block is divided into horizontal BT, horizontal TT, horizontal upper AT, and horizontal lower AT.

[0245] FIGS. 23 (25) to (28) illustrate that after the vertical left AT division, each sub-block is divided into horizontal BT, horizontal TT, horizontal upper AT, and horizontal lower AT. FIGS. 23 (29) to (32) illustrates that after the vertical right AT division, each sub-block is divided into horizontal BT, horizontal TT, horizontal upper AT, and horizontal lower AT.

[0246]

[0247] Example 7

[0248] While compression efficiency can be improved by increasing the degree of freedom of division through block division of various structures, the increase in complexity resulting from the enhanced degree of freedom of division may be a problem. Therefore, in one embodiment of the present invention, a method is proposed for transmitting a syntax that adjusts the degree of QT division by considering the relationship between performance and complexity when performing block division based on a QT structure.

[0249] As an example, the encoder may transmit at least one of the syntax (or syntax elements) of the maximum QT block size (MaxQTSize), minimum QT block size (MinQTSize), and maximum QT block splitting depth (MaxQTDepth) to the decoder. Here, the maximum QT block size represents the maximum size of a block on which QT splitting can be performed and may be expressed in the form of a logarithmic scale (e.g., log2). The minimum QT block size represents the minimum size of a block on which QT splitting can be performed and may be expressed in the form of a logarithmic scale (e.g., log2). Additionally, the maximum QT splitting depth indicates how many QT splittings are allowed from the maximum QT block size.

[0250] Additionally, as an embodiment, the encoder may transmit at least one of the syntax (or syntax element) of the maximum BT block size (MaxBTSize), minimum BT block size (MinBTSize), and maximum BT block splitting depth (MaxBTDepth) to the decoder. Here, the maximum BT block size indicates the maximum size of a block for which BT splitting is allowed and may be expressed in the form of a logarithmic scale (e.g., log2). The minimum BT block size indicates the minimum size of a block for which BT splitting is allowed and may be expressed in the form of a logarithmic scale (e.g., log2). Additionally, the maximum BT splitting depth indicates how many BT splittings are allowed from the maximum BT block size.

[0251] Additionally, as an embodiment, the encoder may transmit at least one of the syntax (or syntax element) of the maximum TT block size (MaxTTSize), the minimum TT block size (MinTTSize), and the maximum TT block division depth (MaxTTDepth) to the decoder. Here, the maximum TT block size indicates the maximum size of a block for which TT division is allowed and may be expressed in the form of a logarithmic scale (e.g., log2). The minimum TT block size indicates the minimum size of a block for which TT division is allowed and may be expressed in the form of a logarithmic scale (e.g., log2). Additionally, the maximum TT division depth indicates how many TT divisions are allowed from the maximum TT block size.

[0252] Each syntax information may be transmitted through a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)), or Network Abstract Layer (NAL) header, depending on the type of slice (or tile group), tile, or image component.

[0253]

[0254] Example 8

[0255] In Example 7 described above, syntax for the maximum size or minimum size or maximum allowable depth of QT, BT, and TT segmentation in an image is defined. In the present invention, the block sizes of the coding unit (CU), prediction unit (PU), and transform unit (TU) may or may not be the same.

[0256] In one embodiment of the present invention, when the sizes of CU, TU, and PU are different, the encoder and decoder propose a method of transmitting a syntax that adjusts the degree of division of CU, PU, ​​and TU by considering the relationship between performance and complexity. In the present invention, TU may refer to a unit in which a transformation is performed by dividing into a tree structure (e.g., QT, QTBT, QTBT+TT, etc.) according to the division information signaled from the encoder, or TU may refer to a unit in which a transformation is performed by implicitly dividing according to the size of the transformation kernel (or transformation matrix).

[0257] In one embodiment, when TU represents a unit in which a transformation is performed by dividing into a tree structure according to the division information signaled from the encoder, blocks such as CU and PU may be recursively divided according to the signaled division information.

[0258] In another embodiment, where TU represents a unit in which a transformation is performed by implicitly subdividing it according to the size of the transformation kernel, the encoder and decoder can equally subdivide blocks such as CU and PU based on the size of the stored transformation kernel without signaling of subdivision information. That is, TU subdivision can be determined dependently on the transformation kernel size. For example, if the size of CU or PU is 128x128 and the maximum size of an applicable (or predetermined) transformation kernel is 64x64, the encoder and decoder can perform the transformation after subdividing CU or PU into four 64x64 blocks without separate subdivision information signaling for the transformation. An example of an implicit subdivision method for transformation is described in the figure below.

[0259] FIG. 24 is a diagram illustrating a method for determining whether to split a block based on the maximum conversion block size without syntax signaling, as an embodiment to which the present invention is applied.

[0260] Referring to FIG. 24, the decoder can check whether the current block size is larger than the maximum conversion size (or the maximum size of the conversion kernel, the maximum size of the conversion matrix) (S2401).

[0261] If the current block size is larger than the maximum transform size, the decoder can divide the current block (S2402). In this case, the decoder can divide the current block into units of the maximum transform size. The current block may be divided into units of the maximum transform size only when performing the inverse transform. The decoder can perform the inverse transform on the divided blocks (S2403).

[0262] Meanwhile, in step S2401, if the current block size is not larger than the maximum conversion size, the decoder can perform an inverse conversion on the current block.

[0263] In one embodiment, syntax such as maximum CU block size (MaxCUSize), minimum CU block size (MinCUSize), and maximum CU block division depth (MaxCUDepth) may be transmitted. More specifically, maximum CU block size refers to the size of the maximum block that first divides the image and may be expressed in the form of a log2 scale. This may be used identically to or substituted with syntax such as maximum QT block size (MaxQTSize), minimum QT block size (MinQTSize), and maximum QT block division depth (MaxQTDepth).

[0264] In another embodiment, syntax such as a maximum PU block size (MaxPUSize), a minimum PU block size (MinPUSize), and a maximum PU block division depth (MaxPUDepth) may be transmitted. More specifically, the maximum PU block size refers to the size of the maximum block that first divides the image and may be expressed in the form of a log2 scale. For example, if the maximum CU size is greater than the maximum PU size, the corresponding CU may be implicitly divided up to the maximum PU without signaling information. Specifically, when the maximum PU size is 64x64 and the maximum CU size is 128x128, a 128x128 block may be predicted using four 64x64 PUs. Alternatively, the block may be implicitly divided into four sub-PUs. A 128x64 block may be predicted using two 64x64 PUs. For a 128x32 block, prediction can be performed using two 64x32 PUs or four 32x32 PUs. If the CU and PU are the same, the syntax may not be transmitted. Alternatively, the syntax may be explicitly transmitted.

[0265] In another embodiment, syntax such as a maximum TU block size (MaxTUSize), a minimum TU block size (MinTUSize), and a maximum TU block depth (MaxTUDepth) may be transmitted. More specifically, the maximum TU block size refers to the size of the maximum block that first divides the image and may be expressed in the form of a log2 scale. For example, if the maximum CU size is greater than the maximum TU size, the corresponding CU may be implicitly divided up to the maximum TU without signaling information. Specifically, when the maximum TU size is 64x64 and the maximum CU size is 128x128, a 128x128 block may be transformed using four 64x64 TUs. Alternatively, the block may be implicitly divided into four sub-CUs. A 128x64 block may be transformed using two 64x64 TUs. A 128x32 block can be transformed using two 64x32 TUs or four 32x32 TUs. If the CU and TU are the same, the syntax may not be transmitted. Alternatively, the syntax may be explicitly transmitted.

[0266] In another embodiment, implicit partitioning based on the transformation size may be performed on CUs that have been TT or AT partitioned from a maximum size CU (or CTU). For example, if the maximum size CU is 128x128 and only one TT partition is performed, two 32x128 CUs and one 64x128 CU may be generated. Assuming the maximum size of the transformation kernel is 64x64, the encoder / decoder may, during the transformation, partition the 32x128 CU into two 64x32 TUs or four 32x32 TUs without separate signaling. In this case, the TU partitioning may be determined in units of applicable transformation kernels or predetermined sizes. Also, during the transformation, the encoder / decoder may, without separate signaling, partition the 64x128 CU into two 64x64 TUs.

[0267] Additionally, for example, if the maximum size CU is 128x128 and only one AT split is performed, one 32x128 CU and one 96x128 CU may be generated. Assuming the maximum size of the conversion kernel is 64x64, when performing the conversion, the encoder / decoder may split the 32x128 CU into two 64x32 TUs or four 32x32 TUs without separate signaling. In this case, the TU splitting may be determined by the applicable conversion kernel unit or a predetermined size unit. Also, when performing the conversion, the encoder / decoder may split the 96x128 CU into two 64x64 TUs and two 64x32 TUs, or two 64x64 TUs and four 32x32 TUs without separate signaling.

[0268] Additionally, if the maximum PU size is greater than the maximum TU size, the corresponding PU may be implicitly divided up to the maximum TU without signaling information. Specifically, when the maximum TU size is 64x64 and the maximum PU size is 128x128, a 128x128 block may be transformed using four 64x64 TUs. Alternatively, the block may be implicitly divided into four sub-PUs. A 128x64 block may be transformed using two 64x64 TUs. A 128x32 block may be transformed using two 64x32 TUs or four 32x32 TUs. If the PU and TU are the same, the syntax may not be transmitted. Alternatively, the syntax may be explicitly transmitted.

[0269] In one embodiment, for PUs that have been TT or AT partitioned from a maximum size PU, implicit partitioning based on the conversion size may be performed. For example, if the maximum size PU is 128x128 and only one TT partition is performed, two 32x128 PUs and one 64x128 PU may be generated. Assuming the maximum size of the conversion kernel is 64x64, when performing the conversion, the encoder / decoder may partition the 32x128 PU into two 64x32 TUs or four 32x32 TUs without separate signaling. In this case, the TU partitioning may be determined in units of applicable conversion kernels or predetermined sizes. Also, when performing the conversion, the encoder / decoder may partition the 64x128 PU into two 64x64 TUs without separate signaling.

[0270] Additionally, for example, if the maximum size PU is 128x128 and only one AT split is performed, one 32x128 PU and one 96x128 PU may be generated. Assuming the maximum size of the conversion kernel is 64x64, when performing the conversion, the encoder / decoder may split the 32x128 PU into two 64x32 TUs or four 32x32 TUs without separate signaling. In this case, the TU splitting may be determined by the applicable conversion kernel unit or a predetermined size unit. Also, when performing the conversion, the encoder / decoder may split the 96x128 PU into two 64x64 TUs and two 64x32 TUs, or two 64x64 TUs and four 32x32 TUs without separate signaling.

[0271] Additionally, if the size of the PU is smaller than the minimum TU size, the block may be performed by merging multiple blocks that satisfy the minimum TU. Alternatively, it may not allow the splitting of PUs smaller than the minimum TU size. Specifically, when the minimum TU size is 8x8 and the minimum PU size is 4x4, the transform may be performed by combining four 4x4 blocks to use a single 8x8 TU. Alternatively, the block may implicitly restrict splitting into 4x4 PUs.

[0272] By utilizing the above syntax information, encoder complexity can be reduced, and since the decoder can implicitly derive block partitioning information, a bit reduction effect may be achieved.

[0273] Each syntax information may be transmitted through a Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Slice Header (SH) (or Tile Group Header (TGH)), or Network Abstract Layer (NAL) header, depending on the type of slice (or tile group), tile, or image component.

[0274] The embodiments of the present invention described above have been described separately for convenience of explanation, but the present invention is not limited thereto. In other words, the embodiments 1 to 8 described above may each be performed independently, or one or more embodiments may be performed in combination.

[0275] FIG. 25 is a flowchart illustrating a method for decoding a video signal according to an embodiment to which the present invention is applied.

[0276] Referring to FIG. 25, the explanation focuses on the decoder for convenience of explanation, but the present invention is not limited thereto, and the decoding method of a video signal according to an embodiment of the present invention can be performed identically in the encoder and the decoder.

[0277] When the current coding tree block goes beyond the boundary of the current picture, the decoder divides the current coding tree block into a plurality of coding blocks so that the coding blocks divided from the current coding tree block are included within the current picture (S2501).

[0278] The decoder parses a first syntax element indicating whether the current coding block is divided into multiple sub-blocks when the current coding block satisfies a preset condition (S2502).

[0279] If the first syntax element indicates that the current coding block is divided, the decoder parses a second syntax element indicating whether the current coding block is divided using a quad-tree structure (S2503).

[0280] If the second syntax element indicates that the current coding block is not divided using a quad tree structure, the decoder parses at least one of a third syntax element indicating whether the current coding block is divided using a binary tree structure or a ternary tree structure, and a fourth syntax element indicating the direction of division of the current coding block (S2504).

[0281] The decoder determines the split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element and the fourth syntax element (S2505).

[0282] As described in Example 5 above, the step of dividing the current coding tree block into a plurality of coding blocks is performed by dividing the current coding tree block into a plurality of coding blocks using a binary tree structure or a binary tree structure until a valid region is reached within the current coding tree block, wherein the valid region may represent a region within the current coding tree block that belongs within the boundary of the current picture.

[0283] In addition, as described in Example 5 above, the step of dividing the current coding tree block into a plurality of coding blocks further includes the step of parsing a fifth syntax element indicating whether quad tree division is allowed when the current coding tree block goes beyond the boundary of the current patch, and when the fifth syntax element indicates that quad tree division is not allowed, the current coding tree block can be divided into a plurality of coding blocks using a binary tree structure or a binary tree structure.

[0284] In addition, as described in Example 5 above, the fifth syntax element can be signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a header of a Network Abstract Layer unit.

[0285] In addition, as described in Example 5 above, the step of dividing the current coding tree block into a plurality of coding blocks is performed by dividing the current coding tree block into a plurality of coding blocks using a quad tree structure until a valid region is reached within the current coding tree block, wherein the valid region may represent a region within the current coding tree block that belongs within the boundary of the current picture.

[0286] In addition, as described in Example 5 above, the step of dividing the current coding tree block into a plurality of coding blocks can be performed by dividing the current coding tree block into coding blocks using a quad tree structure when the width of the valid area is greater than the maximum transformation size or the height of the valid area is greater than the maximum transformation size.

[0287] In addition, in one embodiment, if the first syntax element value is 0, the current block is not divided, and if the first syntax element value is 1, the current block may be divided into 4 sub-blocks using a quad tree structure, into 2 sub-blocks using a binary tree structure, or into 3 sub-blocks using a binary tree structure.

[0288] In addition, in one embodiment, the preset condition may be satisfied if the value obtained by adding the width of the current block to the horizontal coordinate of the top-left sample of the current block is less than or equal to the width of the current picture, and the value obtained by adding the height of the current block to the vertical coordinate of the top-left sample of the current block is less than or equal to the height of the current picture.

[0289] In addition, in one embodiment, the decoder may call a coding tree unit syntax for determining the partition structure of the current block. In this case, the step of calling the coding tree unit syntax for the current block may be performed prior to step S1501.

[0290] In addition, in one embodiment, if the first syntax element indicates that the current block is not divided, the decoder may call a coding unit syntax for the decoding process of the current block.

[0291] Additionally, in one embodiment, the decoder may call a coding tree unit syntax for determining the partition structure of a sub-block partitioned from the current block based on the partition mode.

[0292] Additionally, in one embodiment, the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element can be parsed at the same level of syntax.

[0293] FIG. 26 is a drawing illustrating a decoding device for a video signal according to an embodiment to which the present invention is applied.

[0294] In FIG. 26, the decoding device is illustrated as a single block for convenience of explanation, but the decoding device illustrated in FIG. 26 can be implemented as a configuration included in an encoder and / or decoder.

[0295] Referring to FIG. 26, the decoding device implements the functions, processes, and / or methods proposed in FIGS. 1 through 25. Specifically, the decoding device may be configured to include a coding tree block splitting unit (2601), a syntax element parsing unit (2602), and a splitting mode determining unit (2603). Additionally, in one embodiment, the coding tree block splitting unit (2601) and the syntax element parsing unit (2602) may be implemented as components included in the splitting mode determining unit (2603).

[0296] The coding tree block division unit (2601) divides the current coding tree block into a plurality of coding blocks so that when the current coding tree block goes beyond the boundary of the current picture, the coding blocks divided from the current coding tree block are included within the current picture.

[0297] The syntax element parsing unit (2602) parses a first syntax element indicating whether the current coding block is divided into a plurality of sub-blocks when the current coding block satisfies a preset condition.

[0298] The syntax element parsing unit (2602) parses a second syntax element that indicates whether the current coding block is divided using a quad-tree structure when the first syntax element indicates that the current coding block is divided.

[0299] The syntax element parsing unit (2602) parses at least one of a third syntax element indicating whether the current coding block is divided using a binary-tree structure or a ternary-tree structure, and a fourth syntax element indicating the direction of division of the current coding block, when the second syntax element indicates that the current coding block is not divided using a quad tree structure.

[0300] The split mode determination unit (2603) determines the split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element.

[0301] As described in Example 5 above, the coding tree block splitting unit (2601) can divide the current coding tree block into a plurality of coding blocks using a binary tree structure or a binary tree structure until a valid region within the current coding tree block is reached. Here, the valid region may represent a region within the current coding tree block that is within the boundary of the current picture.

[0302] Additionally, as described in Example 5 above, the coding tree block splitting unit (2601) can parse a fifth syntax element indicating whether quad tree splitting is allowed when the current coding tree block goes beyond the boundary of the current partition. If the fifth syntax element indicates that quad tree splitting is not allowed, the current coding tree block can be split into a plurality of coding blocks using a binary tree structure or a binary tree structure.

[0303] In addition, as described in Example 5 above, the fifth syntax element can be signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a header of a Network Abstract Layer unit.

[0304] In addition, as described in Example 5 above, the coding tree block splitting unit (2601) can divide the current coding tree block into a plurality of coding blocks using a quad tree structure until it reaches a valid region within the current coding tree block. Here, the valid region may represent a region within the current coding tree block that is within the boundary of the current picture.

[0305] In addition, as described in Example 5 above, the coding tree block splitting unit (2601) can split the current coding tree block into coding blocks using a quad tree structure when the width of the valid area is greater than the maximum conversion size or when the height of the valid area is greater than the maximum conversion size.

[0306] In addition, in one embodiment, if the first syntax element value is 0, the current block is not divided, and if the first syntax element value is 1, the current block may be divided into 4 sub-blocks using a quad tree structure, into 2 sub-blocks using a binary tree structure, or into 3 sub-blocks using a binary tree structure.

[0307] In addition, in one embodiment, the preset condition may be satisfied if the value obtained by adding the width of the current block to the horizontal coordinate of the top-left sample of the current block is less than or equal to the width of the current picture, and the value obtained by adding the height of the current block to the vertical coordinate of the top-left sample of the current block is less than or equal to the height of the current picture.

[0308] In addition, in one embodiment, the decoder may call a coding tree unit syntax for determining the partition structure of the current block. In this case, the step of calling the coding tree unit syntax for the current block may be performed prior to step S1501.

[0309] In addition, in one embodiment, if the first syntax element indicates that the current block is not divided, the decoder may call a coding unit syntax for the decoding process of the current block.

[0310] Additionally, in one embodiment, the decoder may call a coding tree unit syntax for determining the partition structure of a sub-block partitioned from the current block based on the partition mode.

[0311] Additionally, in one embodiment, the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element can be parsed at the same level of syntax.

[0312] FIG. 27 shows a video coding system to which the present invention is applied.

[0313] A video coding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data in the form of a file or streaming to the receiving device via a digital storage medium or a network.

[0314] The source device may include a video source, an encoding apparatus, and a transmitter. The receiving device may include a receiver, a decoding apparatus, and a renderer. The encoding apparatus may be called a video / image encoding device, and the decoding apparatus may be called a video / image decoding device. The transmitter may be included in the encoding apparatus. The receiver may be included in the decoding apparatus. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0315] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0316] The encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0317] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can extract the bitstream and transmit it to a decoding device.

[0318] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0319] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0320] FIG. 28 shows a structural diagram of a content streaming system as an embodiment to which the present invention is applied.

[0321] Referring to FIG. 28, the content streaming system to which the present invention is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0322] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0323] The bitstream above may be generated by an encoding method or a bitstream generation method to which the present invention is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0324] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server transmits the multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server plays the role of controlling commands and responses between each device within the content streaming system.

[0325] The streaming server may receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content may be received in real time. In this case, to provide a seamless streaming service, the streaming server may store the bitstream for a certain period of time.

[0326] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0327] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0328] As described above, the embodiments described in the present invention may be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in each figure may be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0329] In addition, the decoder and encoder to which the present invention is applied may be included in multimedia broadcasting transmission and reception devices, mobile communication terminals, home cinema video devices, digital cinema video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, Video on Demand (VoD) service providers, Over-the-top video (OTT) devices, internet streaming service providers, 3D video devices, video phone video devices, and medical video devices, and may be used to process video signals or data signals. For example, Over-the-top video (OTT) devices may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, Digital Video Recorders (DVRs), etc.

[0330] Furthermore, the processing method to which the present invention is applied may be produced in the form of a program that is executed by a computer and may be stored on a computer-readable recording medium. Multimedia data having a data structure according to the present invention may also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. Additionally, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method may be stored on a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0331] In addition, embodiments of the present invention may be implemented as a computer program product by program code, and said program code may be executed on a computer by the embodiments of the present invention. said program code may be stored on a carrier readable by a computer.

[0332] The embodiments described above are combinations of the components and features of the present invention in a specific form. Each component or feature should be considered optional unless otherwise explicitly stated. Each component or feature may be implemented in a form not combined with other components or features. Additionally, it is possible to construct embodiments of the present invention by combining some components and / or features. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment. It is obvious that embodiments may be constructed by combining claims that do not have an explicit citation relationship in the claims, or that new claims may be included by amendment after filing.

[0333] Embodiments according to the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, one embodiment of the present invention may be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, etc.

[0334] In the case of implementation by firmware or software, an embodiment of the present invention may be implemented in the form of a module, procedure, function, etc., that performs the functions or operations described above. The software code may be stored in memory and executed by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means already known.

[0335] It is obvious to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential features of the invention. Accordingly, the foregoing detailed description should not be interpreted restrictively in all respects but should be considered exemplary. The scope of the invention shall be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of the invention are included within the scope of the invention.

[0336] The preferred embodiments of the present invention described above are disclosed for illustrative purposes only, and those skilled in the art may modify, change, substitute, or add various other embodiments within the technical scope and concept of the present invention as disclosed in the appended claims.

Claims

1. In the method of decoding the video, A step of dividing the current coding tree block into a plurality of coding blocks such that the coding blocks divided from the current coding tree block are included in the current picture when the current coding tree block goes beyond the boundary of the current picture; A step of parsing a first syntax element indicating whether the current coding block is divided into a plurality of sub-blocks, if the current coding block satisfies a preset condition; A step of parsing a second syntax element indicating whether the current coding block is divided using a quad-tree structure, when the first syntax element indicates that the current coding block is divided; A step of parsing at least one of a third syntax element indicating whether the current coding block is divided using a binary-tree structure or a ternary-tree structure and a fourth syntax element indicating a division direction of the current coding block, when the second syntax element indicates that the current coding block is not divided using a quad-tree structure; and A method for decoding an image, comprising the step of determining a split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element.

2. In paragraph 1, The step of dividing the above current coding tree block into multiple coding blocks is: A method for decoding an image, the method comprising: dividing the current coding tree block into a plurality of coding blocks using a binary tree structure or a ternary tree structure until a valid region within the current coding tree block is reached, wherein the valid region represents a region within the boundary of the current picture within the current coding tree block.

3. In paragraph 1, The step of dividing the above current coding tree block into multiple coding blocks is: If the current coding tree block goes out of the boundary of the current fetcher, further comprising a step of parsing a fifth syntax element indicating whether a quad tree split is allowed, A method for decoding an image, wherein the current coding tree block is divided into a plurality of coding blocks using a binary tree structure or a ternary tree structure, when the fifth syntax element indicates that quad tree division is not allowed.

4. In paragraph 3, A method for decoding an image, wherein the fifth syntax element is signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a header of a Network Abstract Layer unit.

5. In paragraph 1, The step of dividing the above current coding tree block into multiple coding blocks is: A method for decoding an image, the method comprising: dividing the current coding tree block into a plurality of coding blocks using a quad tree structure until a valid region within the current coding tree block is reached, wherein the valid region represents a region within the boundary of the current picture within the current coding tree block.

6. In paragraph 5, The step of dividing the above current coding tree block into multiple coding blocks is: A method for decoding an image, performed by dividing the current coding tree block into coding blocks using a quad tree structure, when the width of the valid region is greater than the maximum transformation size, or the height of the valid region is greater than the maximum transformation size.

7. In a device for decoding video, A coding tree block splitting unit that splits the current coding tree block into a plurality of coding blocks such that the coding blocks split from the current coding tree block are included in the current picture when the current coding tree block goes beyond the boundary of the current picture; If the current coding block satisfies a preset condition, parse a first syntax element indicating whether the current coding block is divided into multiple sub-blocks, If the first syntax element indicates that the current coding block is divided, parse the second syntax element indicating whether the current coding block is divided using a quad-tree structure, A syntax element parsing unit that parses at least one of a third syntax element indicating whether the current coding block is divided using a binary-tree structure or a ternary-tree structure and a fourth syntax element indicating a division direction of the current coding block, when the second syntax element indicates that the current coding block is not divided using a quad-tree structure; and A video decoding device comprising a split mode determining unit that determines a split mode of the current coding block based on at least one of the first syntax element, the second syntax element, the third syntax element, and the fourth syntax element.

8. In paragraph 7, The above coding tree block division part is, A video decoding device that divides the current coding tree block into a plurality of coding blocks using a binary tree structure or a ternary tree structure until a valid region within the current coding tree block is reached, wherein the valid region represents a region within the boundary of the current picture within the current coding tree block.

9. In paragraph 7, The above coding tree block division part is, If the current coding tree block is outside the boundary of the current fetcher, the fifth syntax element is parsed, which indicates whether a quad tree split is allowed. A video decoding device, wherein the current coding tree block is divided into a plurality of coding blocks using a binary tree structure or a ternary tree structure, when the fifth syntax element indicates that quad tree division is not allowed.

10. In paragraph 9, A video decoding device, wherein the fifth syntax element is signaled through a Sequence Parameter Set, a Picture Parameter Set, a Tile Group Header, or a Network Abstract Layer unit header.

11. In paragraph 7, The above coding tree block division part is, A video decoding device that divides the current coding tree block into a plurality of coding blocks using a quad tree structure until a valid region within the current coding tree block is reached, wherein the valid region represents a region within the boundary of the current picture within the current coding tree block.

12. In paragraph 11, The above coding tree block division part is, A video decoding device that divides the current coding tree block into coding blocks using a quad tree structure when the width of the valid area is greater than the maximum transformation size, or when the height of the valid area is greater than the maximum transformation size.

Images