Method of decoding or encoding an image and method of transmitting image data

Abstract

A method of decoding or encoding an image and a method of transmitting image data are provided. The method of decoding an image includes determining a current transform block by splitting a coding tree block based on a tree-based partitioning, the tree-based partitioning including at least one of a quad-tree partitioning and a binary-tree partitioning; determining whether at least one non-zero residual coefficient is included in the current transform block; and in response to determining that at least one non-zero residual coefficient is included in the current transform block, obtaining residual coefficients of the current transform block according to a scan order of the current transform block, wherein the residual coefficients of the current transform block are obtained based on coefficient level information of the current transform block, wherein the coefficient level information includes at least one of a first residual level flag, a parity flag, and a second residual level flag, wherein the parity flag and the second residual level flag are signaled via a bitstream only when the first residual level flag indicates that an absolute value of the residual coefficient is greater than 1.

Description

[0001] This application is a divisional application of Chinese patent application filed on March 27, 2019, with international application number PCT / KR2019 / 003591, entitled "Method and apparatus for processing video signals", and application number 201980035242.5 which entered the Chinese national phase. Technical Field

[0002] This invention relates to methods and apparatus for processing video signals. Background Technology

[0003] Recently, the demand for high-resolution and high-quality images (e.g., high-definition (HD) and ultra-high-definition (UHD) images) has increased across various application areas. However, higher resolution and quality image data involve increased data volume compared to conventional image data. Therefore, transmission and storage costs increase when transmitting image data using media such as conventional wired and wireless broadband networks, or when storing image data using conventional storage media. To address these issues arising from the increasing resolution and quality of image data, efficient image encoding / decoding techniques can be utilized.

[0004] Image compression techniques encompass a variety of methods, including: inter-frame prediction techniques that predict pixel values ​​included in the current image based on previous or subsequent images; intra-frame prediction techniques that predict pixel values ​​included in the current image using pixel information from the current image; and entropy coding techniques that assign short codes to frequently occurring values ​​and long codes to less frequently occurring values. Image data can be effectively compressed using such image compression techniques, and the image data can be transmitted or stored.

[0005] Simultaneously, the demand for high-resolution images has increased, as has the demand for stereoscopic image content as a new image service. Video compression technologies for effectively delivering stereoscopic image content with both high and ultra-high resolution are being discussed. Summary of the Invention

[0006] Technical issues

[0007] The purpose of this invention is to provide a method and apparatus for efficiently splitting encoded / decoded target blocks when encoding / decoding video signals.

[0008] The purpose of this invention is to provide a method and apparatus for splitting an encoding / decoding target block into symmetric or asymmetric blocks when encoding / decoding a video signal.

[0009] The purpose of this invention is to provide a method and apparatus for setting a prediction target block or a transformation target block to be different in size / shape from that of the encoded block when encoding / decoding a video signal.

[0010] The object of this invention is to provide a method and apparatus for adaptively determining the size / shape of sub-blocks for encoding / decoding residual coefficient encoding indicators when encoding / decoding video signals.

[0011] The purpose of this invention is to provide a method and apparatus for setting the absolute value of residual coefficients to be equal to or less than a preset value when encoding / decoding video signals.

[0012] The technical objectives to be achieved by this invention are not limited to the technical problems mentioned above. Furthermore, those skilled in the art will clearly understand from the following description other technical problems not mentioned.

[0013] Technical solution

[0014] According to the present invention, a method and apparatus for decoding video signals are provided, wherein it is determined whether a non-zero residual coefficient is included in the current block, and when it is determined that a non-zero residual coefficient is included in the current block, the residual coefficients included in the current block are decoded according to the scan order of the current block. In this document, the absolute value of a residual coefficient whose scan index is greater than the residual coefficient adjustment index is equal to or less than the residual coefficient adjustment reference value.

[0015] According to the present invention, a method and apparatus for encoding video signals are provided, wherein it is determined whether a non-zero residual coefficient is included in the current block, and when it is determined that a non-zero residual coefficient is included in the current block, the residual coefficients included in the current block are encoded according to the scan order of the current block. In this document, the absolute value of a residual coefficient whose scan index is greater than the residual coefficient adjustment index is equal to or less than the residual coefficient adjustment reference value.

[0016] In the method and apparatus for encoding / decoding video signals according to the present invention, the residual coefficient adjustment index can be determined based on at least one of the current block size and the position of the last valid residual coefficient.

[0017] In the method and apparatus for encoding / decoding video signals according to the present invention, the residual coefficient adjustment reference value can be 1, the absolute value of the residual coefficient can be set to be the same as the value of the residual level indicator, and the residual level indicator can indicate whether the absolute value of the residual coefficient is greater than 0.

[0018] In the method and apparatus for encoding / decoding video signals according to the present invention, the residual coefficient adjustment reference value is set differently for each slice.

[0019] In the method and apparatus for encoding / decoding video signals according to the present invention, the residual coefficient adjustment reference value can be set to be the same as the absolute value of the residual coefficient where the scan index is the residual coefficient adjustment index.

[0020] In the method and apparatus for encoding / decoding video signals according to the present invention, the determination of whether non-zero residual coefficients are included in the current block can be performed based on the residual coefficient encoding indicator of the current block.

[0021] According to the present invention, a method for decoding an image is provided, comprising: determining a current transform block by segmenting a coding tree block based on tree-based segmentation, wherein the tree-based segmentation includes at least one of quadtree segmentation and binary tree segmentation; determining whether the current transform block includes at least one non-zero residual coefficient; and, in response to determining that the current transform block includes at least one non-zero residual coefficient, obtaining the residual coefficients of the current transform block according to the scan order of the current transform block, wherein the residual coefficients of the current transform block are obtained based on coefficient level information of the current transform block, wherein the coefficient level information includes at least one of a first residual level flag, a parity flag, and a second residual level flag, wherein the residual... The absolute value of the coefficient is determined by the following steps: determining whether the absolute value of the residual coefficient is greater than 1 based on a first residual level flag; in response to the first residual level flag indicating that the absolute value of the residual coefficient is greater than 1; determining whether the absolute value of the residual coefficient is even or odd based on a parity flag; and determining whether the absolute value of the residual coefficient is greater than 3 based on a second residual level flag, wherein the parity flag and the second residual level flag are signaled via a bit stream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1; and wherein, in response to the second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3, the absolute value of the residual coefficient is determined to be 2 or 3 according to the value of the parity flag.

[0022] According to the present invention, a method for encoding an image is provided, comprising: determining a current transform block by segmenting a coding tree block based on tree-based segmentation, wherein the tree-based segmentation includes at least one of quadtree segmentation and binary tree segmentation; determining whether the current transform block includes at least one non-zero residual coefficient; and, in response to determining that the current transform block includes at least one non-zero residual coefficient, encoding the residual coefficients of the current transform block according to the scan order of the current transform block to generate coefficient level information of the current transform block, wherein the coefficient level information includes a first residual level flag, a parity flag, and a second residual level flag. At least one of a first residual level flag and a second residual level flag, wherein a first residual level flag indicating whether the absolute value of the indicated residual coefficient is greater than 1 is encoded into the bitstream, wherein a parity flag indicating whether the absolute value of the residual coefficient is even or odd and a second residual level flag indicating whether the absolute value of the residual coefficient is greater than 3 are further encoded into the bitstream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1, and wherein a second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3 is encoded into the bitstream in response to the absolute value of the residual coefficient being 2 or 3.

[0023] According to the present invention, a method for transmitting image data is also provided, comprising: obtaining a bit stream of image data, wherein the bit stream is generated by the following steps: determining a current transform block by segmenting a coding tree block based on tree-based segmentation, the tree-based segmentation including at least one of quadtree segmentation and binary tree segmentation; determining whether at least one non-zero residual coefficient is included in the current transform block; and, in response to determining that at least one non-zero residual coefficient is included in the current transform block, encoding the residual coefficients of the current transform block according to the scan order of the current transform block; and transmitting the bit stream, wherein the bit stream includes coefficients generated by encoding the residual coefficients of the current transform block. The residual level information includes at least one of a first residual level flag, a parity flag, and a second residual level flag, wherein the first residual level flag indicates whether the absolute value of the residual coefficient is greater than 1, the parity flag indicates whether the absolute value of the residual coefficient is even or odd, and the second residual level flag indicates whether the absolute value of the residual coefficient is greater than 3, wherein the parity flag and the second residual level flag are encoded into the bitstream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1, and wherein, in response to the absolute value of the residual coefficient being 2 or 3, the second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3 is encoded into the bitstream.

[0024] The features briefly outlined above are merely illustrative aspects of the invention as described in the following detailed description and are not intended to limit the scope of the invention.

[0025] Beneficial effects

[0026] According to the present invention, encoding / decoding efficiency can be improved by effectively splitting the encoding / decoding target block.

[0027] According to the present invention, encoding / decoding efficiency can be improved by splitting the encoding / decoding target block into symmetric or asymmetric type blocks.

[0028] According to the present invention, setting the size / shape of the predicted target block or the transformed target block to be different from the size / shape of the encoded block can improve encoding / decoding efficiency.

[0029] According to the present invention, the size / shape of the sub-blocks used to encode / decode the residual coefficient encoding indicator is adaptively determined, thereby improving encoding / decoding efficiency.

[0030] According to the present invention, setting the absolute value of the residual coefficient to a preset value or smaller can improve encoding / decoding efficiency.

[0031] The effects obtained by the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description. Attached Figure Description

[0032] Figure 1 This is a block diagram illustrating an apparatus for encoding video according to an embodiment of the present invention.

[0033] Figure 2 This is a block diagram illustrating an apparatus for decoding video according to an embodiment of the present invention.

[0034] Figure 3 This is a diagram showing the partitioning mode candidates that can be applied to a coding block when encoding the coding block via inter-frame prediction.

[0035] Figure 4 This is a diagram illustrating an example of hierarchical partitioning of coded blocks based on a tree structure according to an embodiment of the present invention.

[0036] Figure 5 This is a diagram illustrating partitioning types that allow binary tree-based partitioning according to an embodiment of the present invention.

[0037] Figure 6 This is a diagram illustrating an example of binary tree-based partitioning that only allows predetermined types.

[0038] Figure 7 This is a diagram illustrating an example of information related to the allowed number of binary tree partitions in the encoding / decoding process according to an embodiment of the present invention.

[0039] Figure 8This is a flowchart illustrating the process of determining the partition type of a coding unit based on partition information encoded in a hierarchical manner.

[0040] Figure 9 This is a flowchart illustrating the process of determining the partitioning type of non-square-shaped coding units based on partitioning information.

[0041] Figure 10 The partitioning types of coded blocks based on asymmetric binary tree partitioning are shown.

[0042] Figure 11 This is a diagram illustrating examples of dividing a coding block into multiple coding blocks based on quadtree partitioning, symmetric binary tree partitioning, and asymmetric binary tree partitioning.

[0043] Figure 12 This is a diagram illustrating the partitioning types of coded blocks based on quadtrees.

[0044] Figure 13 This is a diagram showing the quadtree partitioning types using multiple vertical lines or multiple horizontal lines.

[0045] Figure 14 This is a diagram illustrating the types of ternary tree partitioning.

[0046] Figure 15 This is a diagram showing another type of partitioning of coded blocks generated as a result of ternary tree partitioning.

[0047] Figure 16 This is a diagram illustrating the partitioning types applicable to coded blocks.

[0048] Figure 17 This is a diagram showing the transformation unit of the coded block generated by the tritree partitioning.

[0049] Figure 18 This is a flowchart illustrating the process of obtaining residual coefficients according to an embodiment of the present invention.

[0050] Figure 19 This is a diagram illustrating an example of encoding residual coefficient encoding indicators for each sub-block.

[0051] Figure 20 and Figure 21 This is a diagram illustrating the coding aspects of coded_sub_block_flag for ultra-small asymmetric coding units.

[0052] Figure 22 This is a diagram illustrating an example of encoding residual coefficient encoding indicators for each sub-block containing a preset number of samples.

[0053] Figure 23This is a graph showing an example of adjusting the values ​​of the residual coefficients.

[0054] Figure 24 This is a flowchart illustrating the process of obtaining residual samples according to an embodiment of the present invention. Detailed Implementation

[0055] Various modifications can be made to this invention, and various embodiments of the invention exist. Examples of these embodiments will now be provided with reference to the accompanying drawings, and examples of these embodiments will be described in detail. However, the invention is not limited thereto, and the exemplary embodiments can be interpreted as including all modifications, equivalents, or alternatives within the technical concept and scope of the invention. In the described drawings, similar reference numerals refer to similar elements.

[0056] The terms 'first,' 'second,' etc., used in this specification may be used to describe various components, but these components should not be construed as limited to these terms. These terms are used only to distinguish one component from other components. For example, without departing from the scope of the invention, a 'first' component may be referred to as a 'second' component, and a 'second' component may similarly be referred to as a 'first' component. The term "and / or" includes a combination of multiple items or any one of multiple items.

[0057] It will be understood that in this specification, when an element is simply referred to as 'connected to' or 'coupled to' another element rather than 'directly connected to' or 'directly coupled to' another element, the element may be 'directly connected to' or 'directly coupled to' another element, or the element may be connected to or coupled to another element with other elements in between. In contrast, it should be understood that when an element is referred to as 'directly coupled to' or 'directly connected to' another element, there are no intermediate elements.

[0058] The terminology used in this specification is for describing particular embodiments only and is not intended to limit the invention. Expressions used in the singular include expressions in the plural unless the expression has a distinct meaning in the context. It should be understood in this specification that terms such as “comprising”, “having”, etc., are intended to indicate the presence of features, numbers, steps, actions, elements, portions, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, elements, portions, or combinations thereof may be present or added.

[0059] Preferred embodiments of the invention will be described in detail below with reference to the accompanying drawings. In the drawings, the same constituent elements are indicated by the same reference numerals, and repeated descriptions of the same elements will be omitted.

[0060] Figure 1This is a block diagram illustrating an apparatus for encoding video according to an embodiment of the present invention.

[0061] Reference Figure 1 The apparatus 100 for encoding video may include: an image segmentation module 110, prediction modules 120 and 125, a transformation module 130, a quantization module 135, a rearrangement module 160, an entropy coding module 165, an inverse quantization module 140, an inverse transformation module 145, a filter module 150, and a memory 155.

[0062] Figure 1 The components shown are illustrated independently to represent different functional features in an apparatus for encoding video. Therefore, it is not implied that each component comprises a separate hardware or software unit. In other words, for convenience, each component includes each of the listed components. Thus, at least two components of each component can be combined to form a single component, or a single component can be divided into multiple components to perform each function. Embodiments combining each component and embodiments dividing a component are also included within the scope of this invention without departing from its spirit.

[0063] Furthermore, some components of the invention may not be essential for performing the basic functions of the invention, but rather optional components that only improve the performance of the invention. The invention can be implemented by excluding components used to improve performance and including only the essential components for realizing the essence of the invention. Structures that exclude optional components used only to improve performance and include only the essential components are also included within the scope of the invention.

[0064] Image segmentation module 110 can divide an input image into one or more processing units. Here, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Image segmentation module 110 can divide an image into a combination of multiple coding units, prediction units, and transform units, and can encode the image by selecting a combination of coding units, prediction units, and transform units using a predetermined criterion (e.g., a cost function).

[0065] For example, an image can be divided into multiple coding units. A recursive tree structure (e.g., a quadtree structure) can be used to divide the image into coding units. A coding unit rooted at an image or the largest coding unit and divided into other coding units can be divided into child nodes corresponding to the number of coding units it was divided into. Coding units that cannot be further divided according to predetermined constraints are used as leaf nodes. That is, assuming that only square partitioning is feasible for a coding unit, a coding unit can be divided into at most four other coding units.

[0066] In the following, in embodiments of the present invention, the encoding unit may refer to a unit that performs encoding or a unit that performs decoding.

[0067] A prediction unit can be one of the partitions in a single coding unit that are divided into square or rectangular shapes of the same size, or a prediction unit can be one of the partitions in a single coding unit that are divided into partitions of different shapes / sizes.

[0068] When a prediction unit undergoing intra-frame prediction is generated based on a coding unit and the coding unit is not the smallest coding unit, intra-frame prediction can be performed without dividing the coding unit into multiple prediction units NxN.

[0069] Prediction modules 120 and 125 may include an inter-frame prediction module 120 that performs inter-frame prediction and an intra-frame prediction module 125 that performs intra-frame prediction. It can be determined whether inter-frame or intra-frame prediction is performed for a prediction unit, and detailed information based on each prediction method (e.g., intra-frame prediction mode, motion vectors, reference image, etc.) can be determined. Here, the processing unit undergoing prediction may differ from the processing unit for which the prediction method and details have been determined. For example, the prediction method, prediction mode, etc., may be determined by the prediction unit, and the prediction may be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform module 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction can be encoded together with the residual value by the entropy coding module 165 and can be transmitted to the device for decoding the video. When using a specific coding mode, the original block can be encoded as is and transmitted to the device for decoding the video without generating a prediction block through prediction modules 120 and 125.

[0070] The inter-frame prediction module 120 can predict prediction units based on information from at least one of the previous or subsequent images of the current image, or in some cases, it can predict prediction units based on information from some coded regions in the current image. The inter-frame prediction module 120 may include a reference image interpolation module, a motion prediction module, and a motion compensation module.

[0071] The reference image interpolation module can receive reference image information from the memory 155 and generate pixel information (integer pixels or smaller) based on the reference image. In the case of luminance pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information (integer pixels or smaller) in 1 / 4-pixel units. In the case of chrominance signals, a 4-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information (integer pixels or smaller) in 1 / 8-pixel units.

[0072] The motion prediction module can perform motion prediction based on a reference image interpolated by the reference image interpolation module. Various methods can be used to compute motion vectors, such as the full search-based block matching algorithm (FBMA), three-step search (TSS), and the new three-step search algorithm (NTS). Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 pixel or 1 / 4 pixel. The motion prediction module can predict the current prediction unit by changing the motion prediction method. Various methods can be used as motion prediction methods, such as skipping methods, merging methods, AMVP (Advanced Motion Vector Prediction) methods, and intra-frame block copying methods.

[0073] The intra-frame prediction module 125 can generate prediction units based on reference pixel information adjacent to the current block, where the reference pixel information is pixel information in the current image. When the neighboring block of the current prediction unit is a block undergoing inter-frame prediction and therefore the reference pixel is a pixel undergoing inter-frame prediction, the reference pixel information of the neighboring block undergoing intra-frame prediction can be used to replace the reference pixel included in the block undergoing inter-frame prediction. That is, when a reference pixel is unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information.

[0074] Intra-frame prediction modes can include directional prediction modes that use reference pixel information based on the prediction direction during prediction and non-directional prediction modes that do not use directional information. The mode used to predict luminance information can be different from the mode used to predict chrominance information, and to predict chrominance information, the intra-frame prediction mode information used to predict luminance information or the predicted luminance signal information can be used.

[0075] When performing intra-prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-prediction can be performed based on the pixels located to the left, upper left, and top of the prediction unit. However, when performing intra-prediction, if the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using NxN partitioning can be used only for the smallest coding unit.

[0076] In intra-frame prediction methods, prediction blocks can be generated after applying an AIS (Adaptive IntraSmoothing) filter to a reference pixel according to the prediction mode. The types of AIS filters applied to the reference pixel can be different. To perform intra-frame prediction, the intra-frame prediction mode of the current prediction unit can be predicted based on the intra-frame prediction modes of prediction units adjacent to it. When predicting the prediction mode of the current prediction unit using mode information predicted by neighboring prediction units, if the intra-frame prediction mode of the current prediction unit is the same as that of neighboring prediction units, predetermined flag information can be used to convey information indicating that the prediction modes of the current prediction unit and those of neighboring prediction units are the same. If the prediction mode of the current prediction unit differs from that of neighboring prediction units, entropy coding can be performed to encode the prediction mode information of the current block.

[0077] Furthermore, residual blocks including information about residual values, which are the differences between the predicted units and the original blocks of the predicted units, can be generated based on the prediction units generated by prediction modules 120 and 125. The generated residual blocks can be input to transformation module 130.

[0078] Transform module 130 can transform the residual block, which includes information about the residual values ​​between the original block and the prediction units generated by prediction modules 120 and 125, using transformation methods such as discrete cosine transform (DCT), discrete sine transform (DST), and kLT. Whether to apply DCT, DST, or kLT to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction units used to generate the residual block.

[0079] The quantization module 135 can quantize the values ​​transformed to the frequency domain by the transformation module 130. The quantization coefficients can vary depending on the blocks or importance of the image. The values ​​calculated by the quantization module 135 can be provided to the inverse quantization module 140 and the rearrangement module 160.

[0080] The rearrangement module 160 can rearrange the coefficients of the quantized residuals.

[0081] The rearrangement module 160 can transform coefficients in two-dimensional block form into coefficients in one-dimensional vector form using a coefficient scanning method. For example, the rearrangement module 160 can use a zigzag scanning method to scan from DC coefficients to coefficients in the high-frequency domain, thereby transforming the coefficients into one-dimensional vector form. Depending on the size of the transform unit and the intra-frame prediction mode, a vertical scan scanning the coefficients in two-dimensional block form in the column direction or a horizontal scan scanning the coefficients in two-dimensional block form in the row direction can be used instead of a zigzag scan. That is, the choice between zigzag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0082] Entropy coding module 165 can perform entropy coding based on the value calculated by rearrangement module 160. Entropy coding can use various coding methods such as exponential Golomb coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).

[0083] The entropy coding module 165 can encode various information from the rearrangement module 160 and the prediction modules 120 and 125 (e.g., residual coefficient information and block type information of coding units, prediction mode information, partitioning unit information, prediction unit information, transform unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.).

[0084] The entropy coding module 165 can entropy code the coefficients of the coding units input from the rearrangement module 160.

[0085] The inverse quantization module 140 can inverse quantize the value quantized by the quantization module 135, and the inverse transform module 145 can inverse transform the value transformed by the transform module 130. The residual value generated by the inverse quantization module 140 and the inverse transform module 145 can be combined with the prediction units predicted by the motion estimation module, motion compensation module and intra-frame prediction module of prediction modules 120 and 125 to generate a reconstruction block.

[0086] The filter module 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0087] Deblocking filters remove block distortion caused by boundaries between blocks in a reconstructed image. To determine whether to perform deblocking, the number of pixels in a block (either rows or columns) can be used as a basis for deciding whether to apply a deblocking filter to the current block. When a deblocking filter is applied to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, horizontal and vertical filtering can be processed in parallel when applying a deblocking filter.

[0088] The offset correction module can correct the offset from the original image on a pixel-by-pixel basis in the deblocked image. To perform offset correction on a specific image, a method that considers the edge information of each pixel can be used to apply the offset, or the following method can be used: dividing the image's pixels into a predetermined number of regions, determining the region to be offset, and applying the offset to the determined region. Adaptive loop filtering (ALF) can be performed based on values ​​obtained by comparing the filtered reconstructed image with the original image. Pixels included in the image can be divided into predetermined groups, a filter to be applied to each of the multiple groups can be determined, and filtering can be performed individually for each group. Information regarding whether ALF is applied and a luminance signal can be transmitted via the encoding unit (CU). The shape and filter coefficients of the filter used for ALF can vary depending on each block. Furthermore, a filter of the same shape (fixed shape) for ALF can be applied regardless of the characteristics of the target block.

[0089] The memory 155 can store reconstructed blocks or reconstructed images calculated by the filter module 150. The stored reconstructed blocks or reconstructed images can be provided to the prediction modules 120 and 125 during inter-frame prediction.

[0090] Figure 2 This is a block diagram illustrating an apparatus for decoding video according to an embodiment of the present invention.

[0091] Reference Figure 2 The device 200 for decoding video may include: an entropy decoding module 210, a rearrangement module 215, an inverse quantization module 220, an inverse transform module 225, prediction modules 230 and 235, a filter module 240, and a memory 245.

[0092] When a video bitstream is input from a device used for encoding video, the input bitstream can be decoded according to the inverse processing of the device used for encoding video.

[0093] The entropy decoding module 210 can perform entropy decoding based on the inverse processing of entropy encoding performed by the entropy encoding module of the device for encoding video. For example, various methods such as exponential Golomb coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) can be applied corresponding to the method performed by the device for encoding video.

[0094] The entropy decoding module 210 can decode information about intra-frame prediction and inter-frame prediction performed by the means for encoding video.

[0095] The rearrangement module 215 can perform rearrangement on the bitstream entropy decoded by the entropy decoding module 210 based on the rearrangement method used in the apparatus for encoding video. The rearrangement module can reconstruct and rearrange coefficients from one-dimensional vector form into coefficients from two-dimensional block form. The rearrangement module 215 can receive information related to the coefficient scan performed in the apparatus for encoding video, and can perform rearrangement based on the scan order performed in the apparatus for encoding video via a method of inverse scanning of the coefficients.

[0096] The inverse quantization module 220 can perform inverse quantization based on quantization parameters received from the means for encoding video and the rearranged coefficients of the block.

[0097] The inverse transform module 225 can perform inverse transforms, namely inverse DCT, inverse DST, and inverse KLT, on the quantization results from the means for encoding video. These inverse transforms are the inverse processing of the transforms (i.e., DCT, DST, and KLT) performed by the transform module. The inverse transform can be performed based on the transmission unit determined by the means for encoding video. The inverse transform module 225 of the means for decoding video can selectively execute transform schemes (e.g., DCT, DST, and KLT) based on multiple pieces of information such as the prediction method, the size of the current block, and the prediction direction.

[0098] Prediction modules 230 and 235 can generate prediction blocks based on information about the prediction blocks received from entropy decoding module 210 and previously decoded block or image information received from memory 245.

[0099] As described above, similar to the operation of a device for encoding video, when performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, intra-frame prediction can be performed on the prediction unit based on the pixels located to its left, upper left, and top. When performing intra-frame prediction, if the size of the prediction unit is different from the size of the transform unit, intra-frame prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-frame prediction using NxN partitioning can be used only for the smallest coding unit.

[0100] Prediction modules 230 and 235 may include a prediction unit determination module, an inter-frame prediction module, and an intra-frame prediction module. The prediction unit determination module can receive various information from the entropy decoding module 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, motion prediction information about the inter-frame prediction method, etc., and can divide the current coding unit into prediction units and determine whether to perform inter-frame prediction or intra-frame prediction on the prediction unit. By using the information required for inter-frame prediction of the current prediction unit received from the means for encoding video, the inter-frame prediction module 230 can perform inter-frame prediction on the current prediction unit based on information from at least one of the previous or subsequent images that include the current image of the current prediction unit. Alternatively, inter-frame prediction can be performed based on information from some pre-reconstructed regions in the current image that includes the current prediction unit.

[0101] To perform inter-frame prediction, it can be determined for the coding unit which of the following modes—skip mode, merge mode, AMVP mode, and inter-block copy mode—will be used as the motion prediction method for the prediction unit included in the coding unit.

[0102] Intra-prediction module 235 can generate prediction blocks based on pixel information in the current image. When the prediction unit is a prediction unit undergoing intra-prediction, intra-prediction can be performed based on intra-prediction mode information of the prediction unit received from the means for encoding video. Intra-prediction module 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation module, and a DC filter. The AIS filter performs filtering on the reference pixels of the current block, and whether to apply the filter can be determined based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using AIS filter information received from the means for encoding video and the prediction mode of the prediction unit. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0103] When the prediction mode of the prediction unit is one in which intra-frame prediction is performed based on pixel values ​​obtained by interpolating reference pixels, the reference pixel interpolation module can interpolate the reference pixels to generate reference pixels that are integers or smaller than integers. When the prediction mode of the current prediction unit is one in which a prediction block is generated without interpolating reference pixels, interpolation of reference pixels is not required. When the prediction mode of the current block is DC mode, the DC filter can generate the prediction block through filtering.

[0104] Reconstructed blocks or reconstructed images can be provided to filter module 240. Filter module 240 may include a deblocking filter, an offset correction module, and an ALF.

[0105] Information regarding whether a deblocking filter should be applied to a corresponding block or image can be received from the means for encoding video, as well as information about which filter, strong or weak, should be applied when applying the deblocking filter. The deblocking filter of the means for decoding video can receive this information from the means for encoding video and can perform deblocking filtering on the corresponding block.

[0106] The offset correction module can perform offset correction on the reconstructed image based on the type and offset value information of the offset correction applied to the image during encoding.

[0107] The ALF can be applied to the coding unit based on information received from the device used to encode the video, such as whether an ALF is applied and ALF coefficient information. The ALF information can be provided as included in a specific parameter set.

[0108] The memory 245 can store reconstructed images or reconstructed blocks for use as reference images or reference blocks, and can provide the reconstructed images to the output module.

[0109] As described above, in embodiments of the present invention, for ease of explanation, the term "encoding unit" is used as a term to refer to a unit used for encoding; however, the encoding unit can also be used as a unit that performs both decoding and encoding.

[0110] Additionally, the current block can represent the target block to be encoded / decoded. Furthermore, depending on the encoding / decoding steps, the current block can represent a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), a prediction block (or prediction unit), etc. In this specification, 'unit' refers to a basic unit used to perform a specific encoding / decoding process, and 'block' can represent a sample array of a predetermined size. The terms 'block' and 'unit' can be used interchangeably if there is no distinction between them. For example, in the embodiments described below, it can be understood that coding block and coding unit have equivalent meanings.

[0111] Images can be encoded / decoded by dividing them into basic blocks of either square or non-square shapes. These basic blocks are called coding tree units (CMUs). A CMU can be defined as the largest allowed coding unit within a sequence or slice. Information about the shape of the CMU, whether it is square or non-square, or its size, can be signaled via sequence parameter sets, image parameter sets, or a slice header. A CMU can be further divided into smaller partitions. For example, if the depth of a partition generated by dividing a CMU is 1, then the depth of a partition generated by dividing a partition of depth 1 can be defined as 2. That is, a partition generated by dividing a CMU into partitions of depth k can be defined as having a depth of k+1.

[0112] A partition of arbitrary size generated by segmenting coding tree units can be defined as a coding unit. A coding unit can be recursively segmented or divided into basic units for performing prediction, quantization, transform, or loop filtering, etc. For example, a partition of arbitrary size generated by segmenting coding units can be defined as a coding unit, or it can be defined as a transform unit or prediction unit, which is a basic unit for performing prediction, quantization, transform, or loop filtering, etc.

[0113] Alternatively, a predicted block of the same or smaller size as the coded block can be determined by predicting the partitioning of the coded block. To predict the partitioning of the coded block, one of the partitioning mode (Part_mode) candidates can be specified. Information indicating the partition index, which specifies one of the partitions, can be transmitted via a bitstream signal. Alternatively, the partition index of the coded block can be determined based on at least one of the current block size, shape, or coding mode. The size or shape of the predicted block can be determined based on the partition mode specified by the partition index. Partition mode candidates can include asymmetric partition types (e.g., nLx2N, nRx2N, 2NxnU, 2NxnD). The number or type of asymmetric partition mode candidates available for the coded block can be determined based on the current block size, shape, or coding mode.

[0114] Figure 3 This demonstrates the partitioning mode candidates that can be applied to a coding block when encoding the coding block via inter-frame prediction.

[0115] When using inter-frame prediction to encode a coded block Figure 3 Any one of the eight partitioning pattern candidates shown is applied to the coded block.

[0116] Conversely, when encoding a coded block using intra-frame prediction, only the square-shaped partitioning is applied to the coded block. That is, when encoding a coded block using intra-frame prediction, the partitioning pattern PART_2Nx2N or PART_NxN is applied to the coded block.

[0117] When the coded block is at its minimum size, the partitioning mode PART_NxN is applied. In this paper, the minimum size of the coded block can be predefined in the encoder and decoder. Alternatively, information about the minimum size of the coded block can be sent via a signal through the bitstream. For example, the minimum size of the coded block can be sent via a signal through the slice header. Therefore, different minimum sizes of coded blocks can be determined for each slice.

[0118] As another example, the partitioning pattern candidates available to a coding block can be set differently based on at least one of the size and shape of the coding block. For example, the number or type of partitioning pattern candidates available to a coding block can be set differently based on at least one of the size and shape of the coding block.

[0119] Alternatively, based on the size or shape of the coding block, the type or number of asymmetric partitioning pattern candidates that the coding block can use can be determined. The number or type of asymmetric partitioning pattern candidates that the coding block can use can be set differently based on at least one of the size and shape of the coding block. For example, when the coding block is in a non-square shape where the width is greater than the height, at least one of the partitioning patterns PART_2NxN, PART_2NxnU, and PART_2NxnD is not used as a partitioning pattern candidate for the coding block. When the coding block is in a non-square shape where the height is greater than the width, at least one of the partitioning patterns PART_Nx2N, PART_nLx2N, and PART_nRx2N is not used as a partitioning pattern candidate for the coding block.

[0120] Typically, the size of the prediction block ranges from 64x64 to 4x4. However, when using inter-frame prediction to encode the coded block, the prediction block is set to a size other than 4x4 in order to reduce the storage bandwidth when performing motion compensation.

[0121] Based on the partitioning pattern, the coded block can be partitioned recursively. That is, based on the partitioning pattern determined by the partitioning index, the coded block can be divided to generate multiple partitions, and each of the multiple partitions can be defined as a coded block.

[0122] The method of recursively dividing coding units will be described in more detail below. In the embodiments described later, coding unit may refer to coding tree unit or coding unit included in coding tree unit. In addition, "partition" generated as a result of dividing coding blocks may refer to "coding block".

[0123] A coding unit can be divided using at least one line. In this paper, the angle of the line dividing the coding unit has a value ranging from 0 degrees to 360 degrees. For example, the angle of the horizontal line can be 0 degrees, the angle of the vertical line can be 90 degrees, the angle of the diagonal line in the upward right direction can be 45 degrees, and the angle of the diagonal line in the upward left direction can be 135 degrees.

[0124] When a coding unit is divided by multiple lines, the lines are at the same angle. Alternatively, at least one of the lines can be at a different angle from another line. Alternatively, the lines dividing a coding tree unit or coding unit can have a predefined angle difference (e.g., 90 degrees).

[0125] Information about the lines that divide the coding unit can be determined by the partitioning pattern. Alternatively, information about at least one of the following: the direction, angle, and position of the lines within the block, and the number of lines can be encoded.

[0126] For ease of description, in the embodiments described later, it is assumed that the coding unit is divided into multiple coding units by using at least one of vertical lines and horizontal lines.

[0127] The number of vertical or horizontal lines used to divide the coding unit can be at least one. For example, a coding unit can be divided into two partitions using one vertical or one horizontal line. Alternatively, a coding unit can be divided into three partitions using two vertical or two horizontal lines. Alternatively, a coding unit can be divided into four partitions using one vertical and one horizontal line, where the width and height of each of the four partitions are half the width and height of the coding unit.

[0128] When a coding unit is divided into multiple partitions using at least one vertical line or at least one horizontal line, these partitions are of uniform size. Alternatively, any partition may have a different size from the others, or the partitions may each have a different size. For example, when a coding unit is divided using two horizontal lines or two vertical lines, the coding unit is divided into three partitions. In this paper, the width ratio or height ratio of the three partitions may be n:2n:n, 2n:n:n, or n:n:2n.

[0129] In the implementation described later, dividing the coding unit into four partitions is referred to as quadtree-based partitioning, and dividing the coding unit into two partitions is referred to as binary tree-based partitioning.

[0130] The accompanying drawings, described later, illustrate the division of coding units using a vertical line and / or a horizontal line. However, within the scope of this invention, it also includes dividing the coding unit into more or fewer partitions than shown in the drawings by using more vertical and / or horizontal lines than those shown in the drawings.

[0131] Figure 4 This is a diagram illustrating an example of hierarchical partitioning of coded blocks based on a tree structure according to an embodiment of the present invention.

[0132] The input video signal is decoded in predetermined block units. The default unit for decoding the input video signal is the coding block. A coding block can be a unit that performs intra / inter-frame prediction, transform, and quantization. Furthermore, a prediction mode (e.g., intra-frame prediction mode or inter-frame prediction mode) is determined on a block-by-block basis, and prediction blocks included in a coding block can share the determined prediction mode. A coding block can be a square or non-square block of any size in the range of 8x8 to 64x64, or it can be a square or non-square block of 128x128, 256x256, or larger.

[0133] Specifically, the coded blocks can be partitioned hierarchically based on at least one of the quadtree partitioning method or the binary tree partitioning method. Here, quadtree-based partitioning means dividing a 2Nx2N coded block into four NxN coded blocks, and binary tree-based partitioning means dividing one coded block into two coded blocks. Even when performing binary tree-based partitioning, square-shaped coded blocks may still exist at lower depths.

[0134] Partitions generated by binary tree-based partitioning can have symmetrical or asymmetrical shapes. Furthermore, coded blocks generated by binary tree-based partitioning can be square blocks or non-square blocks (e.g., rectangular blocks).

[0135] Figure 5 This diagram illustrates the partitioning types of coded blocks based on binary tree partitioning. The partitioning shape of coded blocks based on binary tree partitioning can include symmetric types such as 2NxN (horizontal non-square coding unit) and Nx2N (vertical non-square coding unit), or asymmetric types such as nLx2N, nRx2N, 2NxnU, and 2NxnD. Either symmetric or asymmetric type is allowed as the partitioning type of the coded block.

[0136] For coding units, only square or non-square symmetrical shapes are allowed. In this case, dividing the coding unit into a square partition corresponds to a quadtree CU partition, and dividing the coding unit into a symmetrical non-square partition corresponds to a binary tree partition. Dividing the coding tree unit into square partitions and symmetrical non-square partitions corresponds to quadtree and binary tree CU partitions (QTBT).

[0137] Binary tree-based partitioning can be performed on coded blocks that no longer require quadtree-based partitioning. For coded blocks resulting from binary tree-based partitioning, quadtree-based partitioning may no longer be performed.

[0138] Additionally, the partitioning of child depths can be determined based on the partitioning type of the parent depth partition. For example, when partitioning the parent and child depths based on a binary tree, only binary tree-based partitions of the same type as the parent depth partition are allowed in the child depth partition. For instance, if the binary tree partitioning type of the parent depth partition is 2NxN, the binary tree partitioning type of the child depth partition is also set to 2NxN. Alternatively, if the binary tree partitioning type of the parent depth partition is Nx2N, the partitioning type of the child depth partition is also set to Nx2N.

[0139] Alternatively, the partitioning type of a child depth partition can be determined by considering both the partitioning type of the parent depth partition and the partitioning types of adjacent child depth partitions. Specifically, when partitioning the parent depth partition based on a binary tree, the partitioning type of the child depth partition is determined in such a way that it does not produce the same result as partitioning the parent depth partition based on a quadtree. For example, when the partitioning type of the parent depth partition is 2NxN and the partitioning type of the adjacent child depth partition is Nx2N, the partitioning type of the current child depth partition is not set to Nx2N. This is because having the partitioning type of the current child depth partition as Nx2N would result in the same result as partitioning the parent depth partition using an NxN quadtree. When the partitioning type of the parent depth partition is Nx2N and the partitioning type of the adjacent child depth partition is 2NxN, the partitioning type of the current child depth partition is not set to 2NxN. In other words, when the binary tree partitioning type of the parent depth partition is different from the binary tree partitioning type of the adjacent child depth partition, the binary tree partitioning type of the current child depth partition is set to be the same as the binary tree partitioning type of the parent depth partition.

[0140] Alternatively, the partitioning type of the binary tree with child depth partitioning can be set to be different from that of the binary tree with parent depth partitioning.

[0141] Allowed binary tree partition types can be determined per sequence, per slice, or per coding unit. For example, the allowed binary tree partition types for a coding tree unit can be restricted to 2NxN or Nx2N types. Allowed partition types can be predefined in the encoder or decoder. Alternatively, information about allowed or disallowed partition types can be encoded and transmitted as signals via a bitstream.

[0142] Figure 6 This is a diagram illustrating an example of binary tree-based partitioning that only allows specific types of partitioning.

[0143] Figure 6 (a) shows an example that only allows binary tree-based partitioning of type Nx2N, and Figure 6 (b) shows an example of binary tree-based partitioning that only allows 2NxN type partitions.

[0144] To achieve adaptive partitioning based on quadtrees or binary trees, at least one of the following can be used: information indicating quadtree-based partitioning, information about the size / depth of the coded blocks that allow quadtree-based partitioning, information indicating binary tree-based partitioning, information about the size / depth of the coded blocks that allow binary tree-based partitioning, information about the size / depth of the coded blocks that do not allow binary tree-based partitioning, or information about whether binary tree-based partitioning is performed in the vertical or horizontal direction.

[0145] Additionally, at least one of the following information can be obtained for each coding tree unit or coding unit: the number of allowed binary tree partitions, the depth of allowed binary tree partitions, or the number of allowed depths of binary tree partitions. This information can be encoded at the coding tree unit or coding unit level and can be transmitted to the decoder via a bitstream.

[0146] For example, the syntax 'max_binary_depth_idx_minus1', which indicates the maximum depth allowed for binary tree partitioning, can be encoded / decoded via a bitstream. In this case, max_binary_depth_idx_minus1+1 can indicate the maximum depth allowed for binary tree partitioning.

[0147] Alternatively, the number of allowed binary tree partitions, the depth of allowed binary tree partitions, or the number of allowed binary tree partition depths can be predefined in the encoder and decoder. Alternatively, the number of allowed binary tree partitions, the depth of allowed binary tree partitions, or the number of allowed binary tree partition depths can be determined based on at least one of the sequence or slice index and the size / shape of the coding unit. For example, in a first slice, binary tree partitions can be allowed at one depth, and in a second slice, binary tree partitions can be allowed at two depths.

[0148] Reference Figure 7 The example shown is in Figure 7 In the bitstream, binary tree partitioning has already been performed for coding units of depth 2 and depth 3. Therefore, at least one of the following can be encoded / decoded by the bitstream: information indicating the number of times binary tree partitioning has been performed in the coding unit (i.e., 2 times), information indicating the maximum depth of the partition generated by binary tree partitioning in the coding unit (i.e., depth 3), or information indicating the number of partition depths (i.e., 2 (depth 2 and depth 3)) that have been applied in the coding unit.

[0149] As another example, at least one of the following information can be obtained for each sequence or slice: the number of permitted binary tree partitions, the depth of permitted binary tree partitions, or the number of depths of permitted binary tree partitions. For example, this information can be encoded in units of sequences, images, or slices and transmitted via a bitstream. Therefore, at least one of the following—the number of binary tree partitions in the first slice, the maximum depth of permitted binary tree partitions in the first slice, or the number of depths at which binary tree partitions are performed in the first slice—can differ from that in the second slice. For example, in the first slice, binary tree partitions may be permitted for only one depth, while in the second slice, binary tree partitions may be permitted for two depths.

[0150] As another example, the number of allowed binary tree partitions, the depth of allowed binary tree partitions, or the number of depths of allowed binary tree partitions can be set differently based on the temporal ID of the slice or image. Here, the temporal ID is used to identify each of multiple video layers with scalability having at least one of view, space, time, or quality.

[0151] Information used to determine the partitioning type of coding units can be transmitted via signals from a bitstream. The decoder can decode the information from the bitstream and partition the coding units.

[0152] Figure 8 This is a flowchart illustrating the process of determining the partition type of a coding unit based on partition information encoded in a hierarchical manner.

[0153] At step S810, based on `split_flag`, it can be determined whether to divide the current coding unit into sub-depth coding units. `split_flag` indicates whether to divide the current coding unit into sub-depth coding units. When the value of `split_flag` is true, at step S820, `QT_flag` or `BT_flag` used to determine the partition type of the coding unit is decoded. `QT_flag` indicates whether to perform a quadtree partitioning of the coding unit. `BT_flag` indicates whether to perform a binary tree partitioning of the coding unit. Optionally, only one of `QT_flag` and `BT_flag` can be encoded. When the value of `QT_flag` is true or the value of `BT_flag` is false, at step S830, the coding unit is divided into four coding units. When the value of `QT_flag` is false or the value of `BT_flag` is true, the coding unit is divided into two coding units. When QT_flag is false or BT_flag is true, BT_hor_flag or BT_ver_flag, which indicates the division direction of the coding unit, is decoded at step S840. BT_hor_flag indicates whether the coding unit is divided in the horizontal direction. BT_ver_flag indicates whether the coding unit is divided in the vertical direction. Only one of BT_hor_flag and BT_ver_flag can be selectively encoded. When BT_hor_flag is true or BT_ver_flag is false, the coding unit is divided in the horizontal direction (2NxN) at step S850. When BT_hor_flag is false or BT_ver_flag is true, the coding unit is divided in the vertical direction (Nx2N) at step S860.

[0154] The process of dividing a coding unit into multiple sub-depth coding units and determining the partitioning type of each sub-depth coding unit can be performed recursively. In this paper, it can be set that quadtree partitioning is not allowed for coding units that are not square in shape or for coding units generated as a result of binary tree partitioning. Whether to partition non-square in shape coding units or coding units generated as a result of binary tree partitioning can follow the procedures described below. Figure 9 .

[0155] Figure 9 This is a flowchart illustrating the process of determining the partitioning type of non-square-shaped coding units based on partitioning information.

[0156] At step S910, based on the split_flag, it can be determined whether to divide the current coding unit into sub-depth coding units. When the value of split_flag is true, at step S920, BT_hor_flag or BT_ver_flag, which indicates the division direction of the coding unit, is decoded. Only one of BT_hor_flag and BT_ver_flag can be selectively encoded. When the value of BT_hor_flag is true or the value of BT_ver_flag is false, at step S930, the coding unit is divided in the horizontal direction (2NxN). When the value of BT_hor_flag is false or the value of BT_ver_flag is true, at step S940, the coding unit is divided in the vertical direction (Nx2N).

[0157] In the above implementation, it has been described how to use `split_flag` to determine whether to divide the coded block. As another example, the encoding of `split_flag` can be omitted, and the division of the coded block can be determined based on at least one of `QT_flag` and `BT_flag`. For example, when the value of `QT_flag` is true, the coded block is divided into four partitions. When the value of `QT_flag` is false and the value of `BT_flag` is true, the coded block is divided into two partitions. When the values ​​of both `QT_flag` and `BT_flag` are false, the coded block is not divided.

[0158] like Figure 4 As shown, a first coding block 300 with a partition depth (splitting depth) of k can be divided into multiple second coding blocks based on a quadtree. For example, second coding blocks 310 to 340 can be square blocks with half the width and half the height of the first coding block, and the partition depth of the second coding blocks can be increased to k+1.

[0159] The second coding block 310 with a partitioning depth of k+1 can be partitioned into multiple third coding blocks with a partitioning depth of k+2. The partitioning of the second coding block 310 can be performed by selectively using either a quadtree or a binary tree, depending on the partitioning method. Here, the partitioning method can be determined based on at least one of information indicating quadtree-based partitioning and information indicating binary tree-based partitioning.

[0160] When the second coding block 310 is partitioned based on a quadtree, it can be divided into four third coding blocks 310a, each having half the width and half the height of the second coding block, and the partitioning depth of the third coding blocks 310a can be increased to k+2. In contrast, when the second coding block 310 is partitioned based on a binary tree, it can be divided into two third coding blocks. Here, each of the two third coding blocks can be a non-square block having one of the half the width and half the height of the second coding block, and the partitioning depth can be increased to k+2. The second coding block can be determined as a horizontal or vertical non-square block based on the partitioning direction, and the partitioning direction can be determined based on information about whether the binary tree-based partitioning is performed vertically or horizontally.

[0161] Meanwhile, the second coding block 310 can be determined as a leaf coding block that is no longer partitioned based on a quadtree or binary tree. In this case, the leaf coding block can be used as a prediction block or a transform block.

[0162] Similar to the division of the second coding block 310, the third coding block 310a can be determined as a leaf coding block, or it can be further divided based on a quadtree or a binary tree.

[0163] Simultaneously, the third coding block 310b based on binary tree partitioning can be further partitioned into vertical coding blocks 310b-2 or horizontal coding blocks 310b-3 based on the binary tree, and the partitioning depth of the relevant coding blocks can be increased to k+3. Alternatively, the third coding block 310b can be determined as a leaf coding block 310b-1 that is no longer partitioned based on the binary tree. In this case, coding block 310b-1 can be used as a prediction block or a transform block. However, the above partitioning process can be performed restrictively based on at least one of the following: information about the size / depth of coding blocks that allow quadtree-based partitioning, information about the size / depth of coding blocks that allow binary tree-based partitioning, and information about the size / depth of coding blocks that do not allow binary tree-based partitioning.

[0164] The number of candidate sizes can be limited to a predetermined number, or the size of the coded block in a predetermined unit can have a fixed value. As an example, the size of the coded block in a sequence or image can be forced to be one of 256x256, 128x128, or 32x32. Information indicating the size of the coded block in a sequence or image can be sent via signals through the sequence header or image header.

[0165] As a result of partitioning based on quadtrees and binary trees, the coding unit can be represented as a square or rectangle of arbitrary size.

[0166] As a result of quadtree and binary tree partitioning, the coded blocks that are no longer partitioned can be used as prediction blocks or transform blocks. That is, they can be used as coded blocks, prediction blocks, or transform blocks generated as a result of quadtree or binary tree partitioning. For example, a predicted image can be generated based on each coded block, and a residual signal representing the difference between the original image and the predicted image can be generated based on each coded block. To generate a predicted image based on each coded block, motion information can be determined based on the coded blocks, or an intra-frame prediction mode can be determined based on the coded blocks. Therefore, at least one of skip mode, intra-frame prediction, and inter-frame prediction can be used to encode the coded blocks.

[0167] As another example, by dividing the coding block, prediction blocks or transform blocks smaller than the coding block size can be used.

[0168] The binary tree-based partitioning type of a coded block can be set to allow only symmetric partitions. However, even when objects and backgrounds are separated at block boundaries, coding efficiency is reduced when only symmetric binary partitions are allowed. Therefore, this invention proposes a method to divide coded blocks into asymmetric partitions to improve coding / decoding efficiency.

[0169] As a result of asymmetric binary tree partitioning, a coded block can be divided into two smaller coded blocks. These two coded blocks can have different sizes. Two partitions of the same size can be defined as a symmetric binary tree partition. Two partitions of different sizes can be defined as an asymmetric binary tree partition. Binary tree partitioning types can include at least one of symmetric and asymmetric binary tree partitioning types.

[0170] Figure 10 This is a diagram illustrating the partitioning types of coded blocks based on asymmetric binary tree partitioning.

[0171] A 2Nx2N coded block can be divided into two coded blocks with a width ratio of n:(1-n), or two coded blocks with a height ratio of n:(1-n). In this paper, n can represent a real number greater than 0 and less than 1.

[0172] For example, the width ratio or height ratio of the coded blocks generated as a result of asymmetric binary tree partitioning can be 1:3 or 3:1.

[0173] Specifically, when a WxH-sized coding block is divided vertically, a left partition with a width of W / 4 and a right partition with a width of 3W / 4 are generated. As shown above, the partitioning type where the width of the left partition is less than the width of the right partition can be called an nLx2N binary partition.

[0174] When a WxH coded block is divided vertically, a left partition with a width of 3W / 4 and a right partition with a width of W / 4 are generated. As shown above, a partitioning type where the width of the right partition is less than the width of the partition division can be called an nRx2N binary partition.

[0175] When a WxH coded block is divided horizontally, an upper region with a height of H / 4 and a lower region with a height of 3H / 4 are generated. As shown above, the partitioning type where the height of the upper region is less than the height of the lower region can be called a 2NxnU binary partition.

[0176] When a WxH coded block is divided horizontally, an upper region with a height of 3H / 4 and a lower region with a height of H / 4 are generated. As shown above, the partitioning type where the height of the lower region is less than the height of the upper region can be called a 2NxnD binary partition.

[0177] The width or height ratio of two coded blocks generated as a result of asymmetric binary tree partitioning can be predefined in the encoder and decoder. Alternatively, information indicating the width or height ratio of the two coded blocks can be transmitted via a bitstream signal. Alternatively, the width or height ratio of the two coded blocks can be determined based on at least one of the following: block shape, size, coding mode, scan order, and intra-frame prediction mode.

[0178] The asymmetric binary partition shape of a coded block can be determined based on information transmitted via signals through a bit stream. For example, the partition type of a coded block can be determined based on information indicating the partition direction of the coded block and information indicating whether the size of the first partition generated as a result of partitioning the coded block is smaller than the size of the second partition.

[0179] Information indicating the direction of coded block partitioning can be a one-bit flag indicating whether the coded block is partitioned vertically or horizontally. For example, either `hor_binary_flag` or `ver_binary_flag` can be signaled via a bitstream. `hor_binary_flag` indicates whether the coded block is partitioned horizontally, and `ver_binary_flag` indicates whether it is partitioned vertically. When `hor_binary_flag` is 1 or `ver_binary_flag` is 0, the coded block is partitioned horizontally. When `hor_binary_flag` is 0 or `ver_binary_flag` is 1, the coded block is partitioned vertically.

[0180] The information indicating whether the size of the first partition is less than the size of the second partition can be a 1-bit flag. For example, either `is_left_above_small_part_flag` or `is_right_bottom_small_part_flag` can be sent via a bitstream as a signal. `is_left_above_small_part_flag` indicates whether the size of the left or upper partition, resulting from the partitioning of the coded block, is less than the size of the right or lower partition, and `is_right_bottom_small_part_flag` indicates whether the size of the right or lower partition, resulting from the partitioning of the coded block, is less than the size of the left or upper partition. When the value of `is_left_above_small_part_flag` is 1 or the value of `is_right_bottom_small_part_flag` is 0, the size of the left or upper partition is set to be less than the size of the right or lower partition. When the value of is_left_above_small_part_flag is 0 or the value of is_right_bottom_small_part_flag is 1, the size of the left or upper partition is set to be greater than the size of the right or lower partition.

[0181] For example, a value of 0 for `hor_binary_flag` and a value of 1 for `is_left_above_small_part_flag` indicate an nL x 2N binary partition. A value of 0 for both `hor_binary_flag` and `is_left_above_small_part_flag` indicates an nR x 2N binary partition. Additionally, a value of 1 for both `hor_binary_flag` and `is_left_above_small_part_flag` indicates a 2N x nU binary partition. A value of 1 for both `hor_binary_flag` and `is_left_above_small_part_flag` indicates a 2N x nD binary partition.

[0182] Alternatively, information indicating the width ratio, height ratio, or area ratio between the first and second partitions can be used to determine the size of the first and second partitions.

[0183] Alternatively, index information for determining the asymmetric binary partition shape of the coded block can be transmitted via signals, and the coded block can be partitioned based on the asymmetric binary partition shape indicated by the index information. Different indices can be assigned to the asymmetric binary partition shape, and the index information can indicate any of the indices. The index information can be encoded with a fixed length (i.e., a fixed number of bits) or a variable length. For example, Table 1 below shows the partition index and its corresponding binary code.

[0184] [Table 1]

[0185] Asymmetric split index Binarization nLx2N 0 0 nRx2N 1 10 2NxnU 2 100 2NxnD 3 111

[0186] Whether to use an asymmetric binary tree partitioning method can be determined based on whether quadtree partitioning and symmetric binary tree partitioning methods are used. For example, when it is determined that quadtree partitioning or symmetric binary tree partitioning will not be applied to the coded block, it is determined whether to apply asymmetric binary tree partitioning to the coded block. For example, when both QT_flag and BT_flag values ​​are false, the information indicating whether asymmetric binary tree partitioning is applied to the coded block is encoded / decoded. This information can be a 1-bit flag "asymmetric_binary_tree_flag". A value of 1 for asymmetric_binary_tree_flag indicates that asymmetric binary tree partitioning is applied to the coded block, and a value of 0 indicates that asymmetric binary tree partitioning is not applied to the coded block.

[0187] Alternatively, when determining whether to apply a binary tree partition to a coded block, it is determined whether the partition type of the coded block is symmetric or asymmetric. For example, information indicating the partition type of the coded block is encoded / decoded when `split_flag` is true and `QT_flag` is false, or when `split_flag` is true and `BT_flag` is true. This information can be a 1-bit flag "is_asymmetric_split_flag". A value of 1 for `is_asymmetric_split_flag` indicates that the partition type of the coded block is asymmetric, and a value of 0 for `is_asymmetric_split_flag` indicates that the partition type of the coded block is symmetric.

[0188] Alternatively, different indices can be assigned to symmetric binary partition shapes and asymmetric binary partition shapes, and the partition shape of a coded block can be determined based on index information indicating one of the multiple partition shapes. For example, Table 2 shows the partition indices and the binary codes of their corresponding index information.

[0189] [Table 2]

[0190] Binary split index Binarization 2NxN (horizontal binary split) 0 0 Nx2N (vertical binary split) 1 10 nLx2N 2 110 nRx2N 3 1110 2NxnU 4 11110 2NxnD 5 11111

[0191] A coding block can be divided into multiple coding blocks using quadtree partitioning, symmetric binary tree partitioning, or asymmetric binary tree partitioning.

[0192] Figure 11 This diagram illustrates examples of dividing a coding block into multiple coding blocks based on quadtree partitioning, symmetric binary tree partitioning, and asymmetric binary tree partitioning. (See reference...) Figure 11 It was found that asymmetric binary tree partitioning was performed in the partitioning of depth 2 in the first graph, the partitioning of depth 3 in the second graph, and the partitioning of depth 3 in the third graph.

[0193] The coded blocks generated by asymmetric binary tree partitioning can be configured not to be divided into smaller coded blocks. For example, at least one of the following can be omitted from encoding / decoding: information indicating whether partitioning was performed on the coded blocks generated by asymmetric binary tree partitioning; information indicating whether quadtree partitioning was performed; information indicating whether symmetric binary tree partitioning was performed; information indicating whether asymmetric binary tree partitioning was performed; information indicating the partitioning direction; and information indicating the partitioning shape. That is, for coded blocks generated by asymmetric binary tree partitioning, the encoding / decoding of syntax related to the flags indicating whether quadtree partitioning was performed, the flags indicating whether binary tree partitioning was performed, the flags indicating whether symmetric binary tree partitioning was performed, the flags indicating whether asymmetric binary tree partitioning was performed, the flags indicating the direction of binary tree partitioning, or the index information indicating the partitioning shape can be omitted.

[0194] In picture, slice, or coded tree unit where QTBT is not allowed, asymmetric binary tree partitioning may also be disallowed. Alternatively, information indicating whether asymmetric binary tree partitioning is allowed can be encoded per block, per slice, or per picture and can be sent by signaling. This information can be a 1-bit flag. For example, an `is_used_asymmetric_QTBT_enabled_flag` with a value of 0 can indicate the fact that asymmetric binary tree partitioning is not allowed. Alternatively, after determining whether binary tree partitioning is allowed, it can be determined whether asymmetric binary tree partitioning is allowed. For example, at the picture or slice level, when it is determined that binary tree partitioning is allowed, whether to perform asymmetric binary tree partitioning is determined via `is_used_asymmetric_QTBT_enabled_flag`. Alternatively, at the picture or slice level, when it is determined that binary tree partitioning is not allowed, `is_used_asymmetric_QTBT_enabled_flag` is not sent, but its value is set to 0.

[0195] Therefore, asymmetric binary tree partitioning cannot be applied to coded blocks included in picture, slice, or coding tree unit where QTBT or asymmetric binary tree partitioning is not allowed. Therefore, encoding / decoding information related to asymmetric binary tree partitioning of coded blocks can be omitted. Alternatively, values ​​for the syntax related to asymmetric binary tree partitioning can be set so that asymmetric binary tree partitioning is not applied to coded blocks.

[0196] The allowed partition types of a coding block can be determined based on factors such as the size, shape, partition depth, and partition type of the parent node's coding block. For example, at least one of the partition type, partition shape, and number of partitions can be different between a coding block generated by partitioning a quadtree and a coding block generated by partitioning a binary tree.

[0197] For example, any one of quadtree partitioning, binary tree partitioning, and asymmetric binary tree partitioning can be selectively applied to the coded block generated by quadtree partitioning. That is, when a coded block is generated based on quadtree partitioning, candidates for the partitioning method applicable to that coded block include quadtree partitioning, binary tree partitioning, and asymmetric binary tree partitioning. As a result, at least one partitioning shape among 2Nx2N, Nx2N, 2NxN, NxN, nLx2N, nRx2N, 2NxnU, and 2NxnD can be applied to the coded block. A 2Nx2N partition indicates that the coded block is no longer partitioned. NxN indicates that the coded block is partitioned by a quadtree. Nx2N and 2NxN indicate that the coded block undergoes symmetric binary tree partitioning. nLx2N, nRx2N, 2NxnU, or 2NxnD indicate that the coded block undergoes asymmetric binary tree partitioning.

[0198] For example, asymmetric binary tree partitioning may not be allowed for coded blocks generated by binary tree partitioning. That is, asymmetric partitioning shapes such as nLx2N, nRx2N, 2NxnU, 2NxnD, etc., may not be applied to the coded blocks.

[0199] Various partitioning types of coded blocks have been described using the above embodiments. In the following sections, various other partitioning types besides those described above will be described.

[0200] In addition to dividing the coded block into four square blocks, asymmetric quadtree partitioning (Asymmetric Quadtree CU partitioning) is also permitted. Asymmetric quadtree partitioning is performed by dividing the coded block into two asymmetric shapes using at least one of a horizontal line dividing the coded block into two and a vertical line dividing the coded block into two. For example, the following partitioning shapes can be applied to the coded block: a partitioning shape where the horizontal line divides the coded block asymmetricly while the vertical line divides it symmetrically; a partitioning shape where the horizontal line divides the coded block symmetrically while the vertical line divides it asymmetricly; or a partitioning shape where both the horizontal and vertical lines divide the coded block asymmetricly. As a result of asymmetric quadtree partitioning, the width or height of any partition may differ from the width or height of the other partition.

[0201] Figure 12 This is a diagram illustrating the partitioning types of coded blocks based on quadtrees.

[0202] Figure 12 (a) shows the partition shape in which the coded block is symmetrically divided by horizontal and vertical lines. Figure 12 (b) and Figure 12 (c) shows a partition shape in which the horizontal lines divide the coded block symmetrically, while the vertical lines divide the coded block asymmetrically. Figure 12 (d) and Figure 12 (e) shows a partition shape in which the vertical lines divide the coded block symmetrically, while the horizontal lines divide the coded block asymmetrically.

[0203] Information related to the quadtree partitioning type can be encoded in a bitstream. This information could be index information indicating any of several partitioning shape candidates. For example, different indices could be assigned to... Figure 12 The diagram shows multiple partition shape candidates. Additionally, the quadtree partition type of the coded block can be determined based on index information indicating the index of any one of the multiple partition shape candidates. Alternatively, this information may include an indicator indicating whether the quadtree partition type is symmetric or asymmetric. This indicator can be encoded on a per-block basis. For example, when the indicator value is true, the quadtree partition type of the coded block is determined to be a symmetric partition. Figure 12 (a)). When the indicator value is false, the partition type of the coded block is determined to be asymmetric partition type. Figure 12 (b) Figure 12 (c) Figure 12 (d) and Figure 12Any one of (e)). When the indicator value is false, the index information indicating any one of the asymmetric partition types is further encoded / decoded. Alternatively, when the indicator value is false, at least one of the vertical partition information and the horizontal partition information described below is further encoded / decoded.

[0204] Alternatively, at least one of the vertical partitioning information indicating whether the vertical lines symmetrically divide the coding block and the horizontal partitioning information indicating whether the horizontal lines symmetrically divide the coding block can be encoded / decoded. The quadtree partitioning type of the coding block can be determined based on the vertical partitioning information and at least one of the vertical partitioning information.

[0205] Alternatively, the value of either the horizontal or vertical partitioning information can be determined based on the value of the other. That is, only one of the horizontal or vertical partitioning information can be encoded, and the value of the other can be determined based on the value of the encoded information. For example, the horizontal and vertical partitioning information can have different values. When the horizontal partitioning information indicates an asymmetric partition, the vertical partitioning information is determined to be a symmetric partition. Alternatively, when the vertical partitioning information indicates an asymmetric partition, the horizontal partitioning information is determined to be a symmetric partition.

[0206] When the horizontal partitioning information indicates asymmetric partitioning, at least one of the information regarding the position of the horizontal line that performs the asymmetric partitioning of the coded block and the information regarding the height of the coded block generated by the horizontal partitioning is further encoded / decoded. When the vertical partitioning information indicates asymmetric partitioning, at least one of the information regarding the position of the vertical line that performs the asymmetric partitioning of the coded block and the information regarding the width of the coded block generated by the vertical partitioning is further encoded / decoded.

[0207] In addition to using a quadtree partitioning method with one vertical line and one horizontal line, quadtree partitioning methods with multiple vertical lines or multiple horizontal lines can also be used. For example, the following can be applied to coding blocks: quadtree partitioning types using multiple vertical lines and one horizontal line, or multiple horizontal lines and one vertical line, or quadtree partitioning types using multiple vertical lines or multiple horizontal lines.

[0208] Figure 13 This is a diagram showing the quadtree partitioning types using multiple vertical lines or multiple horizontal lines.

[0209] Quadtree partitioning types can include the following: where the coded block is divided into three blocks using two vertical lines, and then any one of the three resulting blocks is partitioned using a horizontal line. Here, the horizontal line can be used to partition any one of the three blocks in a symmetrical or asymmetrical shape. Alternatively, quadtree partitioning types can include the following: where the coded block is divided into three blocks using two horizontal lines, and then any one of the three resulting blocks is partitioned using a vertical line. Here, the vertical line can be used to partition any one of the three blocks in a symmetrical or asymmetrical shape.

[0210] As in Figure 13 In the example shown, when the coded block is divided using two vertical lines or two horizontal lines, the middle block of the three resulting blocks can be further subdivided. Alternatively, the quadtree partitioning type can include a type where the blocks located at the edges of the coded block are further subdivided.

[0211] The partitions to be further subdivided among the three partitions can be predefined in the encoder and decoder. For example, the width or height ratio of the three partitions generated by dividing the coded block using two horizontal or two vertical lines can be n:2n:n, 2n:n:n, or n:n:2n. The partition with the largest width or height among the three partitions can be identified as the target for further subdivision. Alternatively, information specifying the partitions to be further subdivided among the three partitions (e.g., partition index) can be signaled via a bitstream.

[0212] As in Figure 13 In the example shown, the quadtree partitioning type can include at least one of the following: a partitioning type that divides the coded block by combining multiple vertical lines and one horizontal line, and a partitioning type that divides the coded block by combining multiple horizontal lines and one vertical line. Quadtree partitioning can divide the coded block into two partitions of a first size / first shape and two partitions of a second size / second shape. Alternatively, quadtree partitioning can divide the coded block into one partition of a first size / first shape, one partition of a second size / second shape, and two partitions of a third size / third shape. As above, a quadtree partitioning type where the number of partitions of size or shape is two or three can be defined as a triple-type asymmetric quadtree partition (triple-type asymmetric quadtree CU partitioning).

[0213] Information regarding ternary asymmetric quadtree partitioning can be encoded using a bitstream. This information may include at least one of the following: information indicating whether ternary asymmetric quadtree partitioning is permitted, information indicating whether the coded block has undergone ternary asymmetric quadtree partitioning, and information indicating the partition type of the ternary asymmetric quadtree partitioning. Information indicating the partition type may include at least one of the following: index information indicating any one of a plurality of partition shape candidates, information regarding the number of horizontal or vertical lines, information regarding the position of the horizontal or vertical lines, and information indicating the width ratio or height ratio of the partition.

[0214] Quadtree partitioning represents dividing a coded block into four coded blocks, while binary tree partitioning represents dividing a coded block into two coded blocks. Partition types that divide a coded block into three, five, or more coded blocks can be defined. Dividing a coded block into three partitions can be defined as triple-tree partitioning. Triple-tree partitioning types can include at least one type that partitions the coded block using two vertical lines and another type that partitions the coded block using two horizontal lines. Three non-square partitions can be generated through triple-tree partitioning.

[0215] Figure 14 This is a diagram illustrating the types of ternary tree partitioning.

[0216] The ternary tree partitioning type can include a type that uses two horizontal lines to partition the coded block, or a type that uses two vertical lines to partition the coded block. The width ratio or height ratio of the partitions generated as a result of partitioning the coded block can be n:2n:n, 2n:n:n, or n:n:2n.

[0217] The location of the partition with the largest width or height among the three partitions can be predefined in the encoder and decoder. Alternatively, information indicating the partition with the largest width or height among the three partitions can be sent via a bitstream.

[0218] The coded blocks generated as a result of ternary tree partitioning can be divided into smaller coded blocks. Alternatively, for prediction or transformation, the coded blocks generated as a result of ternary tree partitioning can be divided into smaller prediction blocks or transform blocks. Alternatively, among the three coded blocks generated as a result of ternary tree partitioning, only the largest coded block can be divided into smaller coded blocks.

[0219] As another example, the coded blocks generated as a result of ternary tree partitioning can be set not to be further partitioned.

[0220] Alternatively, it can be configured not to apply at least one of quadtree partitioning, ternary tree partitioning, and binary tree partitioning to a coding block resulting from ternary tree partitioning. For example, it can be configured so that quadtree partitioning and ternary tree partitioning are not allowed for a coding block resulting from ternary tree partitioning. For that coding block, only binary tree partitioning can be allowed.

[0221] Figure 15 This is a diagram showing other partitioning types of coded blocks generated as a result of ternary tree partitioning.

[0222] It can be set so that for coded blocks generated as a result of ternary tree partitioning, at least one of the quadtree partitioning method and the ternary tree partitioning method is not allowed. It can also be set so that for coded blocks generated as a result of ternary tree partitioning, only the binary tree partitioning method is allowed.

[0223] Figure 15 (a) shows the following example, in which a vertical binary partition is applied to two coded blocks 1510 and 1530 with small widths in the coded blocks generated as a result of ternary tree partitioning. Figure 15 (b) shows the following example, in which a horizontal binary partition is applied to the coded block 1520 with the maximum width, which is generated as a result of a ternary tree partition. Figure 15 (c) shows the following example, in which a horizontal binary partition is applied to two coded blocks 1510 and 1530 with small widths in the coded blocks generated as a result of ternary tree partitioning.

[0224] It can be set so that, within the coded blocks generated as a result of ternary tree partitioning, binary tree partitioning along the same direction as the aforementioned ternary tree partitioning is not allowed for the coded block with the largest size. For example, as in... Figure 15 In the example shown in (b), among the coded blocks generated as a result of a ternary tree partition along the vertical direction, for the coded block with the largest width, only a binary tree partition along the vertical direction is allowed. Therefore, for this coded block, the encoding / decoding information indicating the direction of the binary tree partition can be omitted.

[0225] Whether ternary tree partitioning is allowed can be determined based on the width, height, width-to-height ratio, size, or shape of the coded block. For example, ternary tree partitioning may be disallowed if at least one of the following conditions is met: the width and height of the coded block are equal to or greater than a predefined value; the size of the coded block is equal to or greater than a predefined value; and the width-to-height ratio of the coded block is equal to or greater than a predefined value. Alternatively, ternary tree partitioning is allowed only for coded blocks of size M x N. Here, N and M can be natural numbers with the same or different values. For example, the values ​​of N and M can be 4, 8, 16, 32, 64, or greater.

[0226] Alternatively, information indicating the width, height, size, width-to-height ratio, or shape of the coded block that allows ternary tree partitioning can be encoded and transmitted via a bitstream. This information can indicate a maximum or minimum value. For example, ternary tree partitioning is allowed for a coded block when its width, height, size, or width-to-height ratio is equal to or greater than the minimum value indicated by the information, or when its width, height, size, or width-to-height ratio is equal to or less than the maximum value indicated by the information. Alternatively, the size or shape of the block allowing ternary tree partitioning can have predetermined fixed values ​​in the encoder / decoder.

[0227] Alternatively, information indicating whether ternary tree partitioning is allowed can be sent via a bitstream using signals. This information can be sent at the picture level, slice level, tile group level, tile level, or block level. This information can be a 1-bit flag. When a coded block is included in a parent level that does not allow ternary tree partitioning, the encoding and decoding information related to the ternary tree partitioning of the coded block is omitted. When a coded block is included in a parent level that allows ternary tree partitioning, the information related to the partitioning type for encoding / decoding the coded block includes information related to ternary tree partitioning.

[0228] Information indicating whether a coded block has been partitioned into three coded blocks can be sent via a bitstream using signals. This information can be a 1-bit flag. For example, the TT_flag indicating whether a coded block has been partitioned into three coded blocks can be encoded / decoded. A true TT_flag indicates that the coded block has been partitioned into three coded blocks. A false TT_flag indicates that the coded block has not been partitioned into three coded blocks.

[0229] Alternatively, you can use QT_flag or BT_flag to determine whether a ternary tree partition is applied to the coded block. For example, a QT_flag with a true value can indicate that the coded block is partitioned into a quadtree, while a QT_flag with a false value can indicate that the coded block is partitioned into a ternary tree.

[0230] Alternatively, a true BT_flag indicates that the coded block is partitioned by a binary tree, while a false BT_flag indicates that the coded block is partitioned by a ternary tree. BT_flag is encoded / decoded when the QT_flag is false, or when the coded block is a non-square partition and the split_flag is 1.

[0231] When ternary tree partitioning is applied to a coded block, at least one of the following is further signaled: information indicating the partitioning direction of the coded block and information indicating the size of the partitions generated as a result of partitioning the coded block. The information indicating the partitioning direction can indicate whether the partitioning direction of the coded block is horizontal or vertical. When the partitioning direction of the coded block is horizontal, the coded block is divided into three coded blocks by two horizontal lines. When the partitioning direction of the coded block is vertical, the coded block is divided into three coded blocks by two vertical lines.

[0232] When a coded block is partitioned by a ternary tree, the partitions within the coded block share motion information, merging candidates, reference samples, intra-prediction modes, etc., depending on the size or shape of the coded block. For example, when the current coded block is partitioned by a ternary tree and the size or shape of the current coded block meets predetermined conditions, the coded blocks within the current coded block share at least one of the following: spatial neighbor block candidates for inter-frame prediction, temporal neighbor block candidates for inter-frame prediction, reference samples for intra-frame prediction, and intra-prediction modes. Alternatively, it can be configured such that only some coded blocks within the current coded block share information, while the remaining coded blocks do not share information.

[0233] In summary, the methods for partitioning coded blocks can include at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning. Quadtree partitioning can include at least one of symmetric quadtree partitioning, asymmetric quadtree partitioning, and ternary asymmetric quadtree partitioning. Binary tree partitioning can include at least one of symmetric binary tree partitioning and asymmetric binary tree partitioning. The method of partitioning coded blocks using at least one of the above-described quadtree partitioning, binary tree partitioning, and ternary tree partitioning methods can be called multi-tree partitioning (or multiple-tree partitioning) methods.

[0234] Figure 16 This is a diagram illustrating the partitioning types applicable to coded blocks.

[0235] When symmetric quadtree partitioning, symmetric binary tree partitioning, asymmetric binary tree partitioning, and ternary tree partitioning are allowed, the partitioning type of the coding block is determined as follows: Figure 16Any one of the nine classification types shown.

[0236] Whether quadtree partitioning, binary tree partitioning, and ternary tree partitioning are allowed can be determined at the sequence level, image level, slice level, tile group level, or tile level. Information indicating whether each partitioning method is allowed can be transmitted via a bitstream signal. This information can be a 1-bit flag. For example, at least one of the following can be transmitted via a bitstream signal: `isUseQuadTreeFlag` indicating whether quadtree partitioning is allowed, `isUseBinaryTreeFlag` indicating whether binary tree partitioning is allowed, and `isUseTripleTreeFlag` indicating whether ternary tree partitioning is allowed. Depending on whether each partitioning method is allowed, quadtree partitioning, binary tree partitioning, and ternary tree partitioning can be used to partition the coded blocks included in the parent level, or one or two of these partitioning methods can be used.

[0237] Alternatively, one or more of the following partitioning methods—quadtree partitioning, binary tree partitioning, and ternary tree partitioning—can be allowed by default, and the remaining partitioning methods can be selectively allowed. For example, quadtree partitioning can be allowed by default, but binary tree partitioning or ternary tree partitioning can be selectively allowed. Alternatively, quadtree and ternary tree partitioning can be allowed by default, but binary tree partitioning can be selectively allowed. Alternatively, quadtree and binary tree partitioning can be allowed by default, but ternary tree partitioning can be selectively allowed. The information indicating whether binary tree partitioning or ternary tree partitioning is allowed can be a 1-bit flag. For example, at least one of isUseBinaryTreeFlag indicating whether binary tree partitioning is allowed and isUseTripleTreeFlag indicating whether ternary tree partitioning is allowed can be sent via a bitstream using signals. This information can be sent at the sequence header, image level, slice level, tile group level, or tile level using signals. For example, a value of 1 for `isUseBinaryTreeFlag` indicates that the coded blocks within the sequence can be partitioned using a binary tree partitioning method. A value of 0 for `isUseBinaryTreeFlag` indicates that the coded blocks included in the sequence cannot be partitioned using a binary tree partitioning method. When binary tree partitioning is not allowed, encoding / decoding information related to the binary tree partitioning of the coded blocks is omitted.

[0238] Alternatively, a value of 1 for isUseTripleTreeFlag indicates that the coded blocks included in the sequence can be partitioned using the ternary tree partitioning method. A value of 0 for isUseTripleTreeFlag indicates that the coded blocks included in the sequence cannot be partitioned using the ternary tree partitioning method. When the ternary tree partitioning method is not allowed, encoding / decoding information related to the ternary tree partitioning of the coded blocks is omitted.

[0239] The coded blocks generated as a result of quadtree partitioning, binary tree partitioning, or ternary tree partitioning can be used as the basis for prediction or transformation. That is, coded blocks can be set as prediction blocks or transformation blocks, and prediction, transformation, or quantization can be performed on the coded blocks.

[0240] Alternatively, the size and / or shape of at least one of the prediction block and the coding block can be set to be different from the size and / or shape of the coding block. For example, by partitioning the coding block, a prediction block or transform block smaller than the size of the coding block can be generated. Alternatively, multiple coding blocks can be set as a single prediction block or transform block. For example, the transform unit of the coding block generated as a result of ternary tree partitioning can be determined as the parent partitioning region. The parent partitioning region can be a non-square or square region that includes at least one of the three blocks generated as a result of ternary tree partitioning. That is, a parent node block including the coding block can be set at the transform unit of the coding block.

[0241] Figure 17 This is a diagram showing the transformation unit of the coded block generated by the tritree partitioning.

[0242] exist Figure 17 In this context, the transformation units CU0, CU1, and CU2 generated as a result of ternary tree partitioning can be set as parent node blocks including CU0, CU1, and CU2. That is, transformations and / or quantizations of CU0, CU1, and CU2 can be performed based on parent node blocks including CU0, CU1, and CU2.

[0243] exist Figure 17 In this context, the transformation units of CU3, CU4, and CU5, generated as a result of ternary tree partitioning, can be set as parent node blocks including CU3, CU4, and CU5. That is, transformations and / or quantizations of CU3, CU4, and CU5 can be performed based on parent node blocks including CU3, CU4, and CU5.

[0244] As described above, multiple coded blocks can be set up as a single transform unit. That is, a transform process can be performed to set up a group of multiple coded blocks as a single transform unit. Transformation processing can be performed on coded blocks generated by binary or quadtree partitioning, in addition to blocks generated by ternary tree partitioning. The transform process can be performed considering at least one of the following: the shape of the coded block, the size of the coded block, the partitioning depth of the coded block, and the partitioning type of the parent node block. For example, a transform process is performed when at least one of the following conditions is met: the size of the coded block is the minimum size; the partitioning depth of the coded block is the maximum partitioning depth; the coded block is not square; and the size of the coded block is equal to or less than a predetermined size. A transform process is performed when at least one of the multiple coded blocks meets the condition. The minimum size or maximum partitioning depth can be predefined in the encoder and decoder. Alternatively, information for determining the minimum size or maximum partitioning depth can be transmitted via a bitstream signal. A predetermined size can be preset in the encoder and decoder.

[0245] Alternatively, the transformation process can be configured to be performed when the shape of the encoded block is a preset shape in the encoder and decoder. The preset shape can be a square shape (e.g., 2x2 or 4x4) or a rectangular shape (e.g., 2x4, 4x2, 2x8 or 8x2).

[0246] Alternatively, information indicating whether to perform transformation processing can be transmitted via a bitstream signal. For example, a flag indicating whether to perform transformation processing on any of a plurality of coded blocks can be transmitted via a bitstream signal. Alternatively, information specifying the first and last coded blocks for which transformation processing will be performed can be transmitted via a bitstream signal.

[0247] Alternatively, information indicating the conditions for performing transformation processing can be transmitted via a bit stream using signals. This information may be at least one of the shape, size, and depth of the coded block that allows transformation processing.

[0248] Prediction can be performed on each of multiple coded blocks, and transformation or quantization can be performed on transform blocks that include coded blocks.

[0249] The encoder can perform transformation and / or quantization on residual samples (or residual signals) based on each predetermined block. The unit block on which the transformation and / or quantization is performed can be defined as a transform block. As a result of performing the transformation and / or quantization on the residual samples, residual coefficients can be generated. Residual coefficients can include at least one of the following: transform coefficients generated by transforming the residual samples, quantized transform coefficients generated by quantizing the transform coefficients, and quantized residual samples generated by quantizing residual samples with skipped transformations. Different transform blocks can be set for each color component. For example, the size of the transform block for the luminance component and the size of the transform block for the chrominance component can be different. The luminance component, the first chrominance component Cb, and the second chrominance component Cr can be transformed and / or quantized independently.

[0250] Transform blocks can be square or non-square. For example, transform blocks can be square, such as 4x4, 8x, 16x16, 32x32, 64x64, etc., or non-square, such as 4x8, 8x4, 8x16, 16x8, 16x32, 32x16, 32x64, 64x32, 4x16, 4x32, 8x32, etc.

[0251] The coded blocks generated by multi-way tree partitioning can be set as transform blocks. Alternatively, the coded blocks can be divided into smaller transform blocks. Alternatively, transform blocks can be generated by merging multiple coded blocks. The number, size, and / or shape of transform blocks included in a coded block, or the number, size, and / or shape of coded blocks included in a transform block, can be determined based on the minimum size of transform blocks predefined in the encoder and decoder, or based on the size of the minimum block used to scan transform coefficients.

[0252] At least one of the following information—whether a coded block is used as a transform block, information for dividing a coded block into transform blocks, and information for merging coded blocks into transform blocks—can be transmitted via a bitstream using signals. This information can be transmitted via signals at the sequence header, picture header, slice header, tile group header, or at the block level (e.g., coded block or transform block).

[0253] The decoder can decode the residual coefficients from the bitstream received from the encoder, and can perform at least one of inverse quantization and inverse transform on the decoded residual signal to decode the residual sample (or residual signal). The process of generating the residual signal by decoding the residual coefficients and performing at least one of inverse quantization and inverse transform on the decoded residual signal can be defined as "residual coefficient decoding".

[0254] The residual coefficient decoding process will be described in detail below.

[0255] Figure 18 This is a flowchart illustrating the process of obtaining residual coefficients according to an embodiment of the present invention.

[0256] The scan order of the current block can be determined (S1810). The scan order may include at least one of diagonal scan, Z scan, zigzag scan, horizontal scan, or vertical scan. The scan order of the current block may be determined based on at least one of the size of the current block, the shape of the current block, the encoding mode of the current block, or the intra-frame prediction mode of the current block.

[0257] Information indicating the presence of non-zero residual coefficients within the current block can be encoded / decoded (S1820). Based on this information, it can be determined whether to encode / decode the residual coefficients of the current block (S1830).

[0258] This information may include a residual coefficient coded_block_flag (CBF) indicating the presence of residual coefficients within the current block. The residual coefficient coded_block_flag can indicate the presence of non-zero transform coefficients within the block. For example, a residual coefficient coded_block_flag with a value of 0 indicates the absence of non-zero residual coefficients within the block, while a residual coefficient coded_block_flag with a value of 1 indicates the presence of at least one non-zero residual coefficient within the block. The residual coefficient coded_block_flag can be encoded for each of the luma and chroma components, and the residual coefficient coded_block_flag can be signaled.

[0259] The residual coefficient encoding indicator may include at least one of an indicator signaled in a first-level block (e.g., "rqt_root_cbf") and an indicator signaled in a second-level block included in a first-level block (e.g., "coded_sub_block_flag", CSBF). The first-level block indicates a transform block, a coding block, or a coding tree block. The second-level block indicates a sub-block included in a second-level block.

[0260] For example, `rqt_root_cbf` can indicate whether non-zero residual coefficients should be included in the current block. The decoder can determine whether to decode the residual coefficients based on the value of `rqt_root_cbf`. For example, when the value of `rqt_root_cbf` is 0, residual coefficient decoding is not performed on the current block (e.g., the current transform block), and all values ​​of the residual samples within the current block are set to 0. Conversely, when the value of `rqt_root_cbf` is 1, residual coefficient decoding is performed within the current block.

[0261] Encoding / decoding the `coded_sub_block_flag` is performed when the current block size is larger than a predefined size. For example, the predefined size could be 4x4, 2x8, or 8x2. `coded_sub_block_flag` can indicate whether non-zero transform coefficients are included within the sub-block. For example, `coded_sub_block_flag` can be encoded on a per-sub-block basis and can be signaled, where the sub-block is 4x4 in size. A value of 0 for `coded_sub_block_flag` indicates that no non-zero residual coefficients exist within the sub-block, while a value of 1 indicates that at least one non-zero residual coefficient exists within the sub-block.

[0262] Figure 19 This is a diagram illustrating an example of encoding residual coefficient encoding indicators for each sub-block.

[0263] First, the 16x16 transform block includes non-zero residual coefficients, and therefore the rqt_root_flag of the transform block can be set to the value 1.

[0264] The `coded_sub_block_flag` can be set for each sub-block within a 4x4 sub-block included in the transform block. When a non-zero residual coefficient exists within a 4x4 sub-block, the `coded_sub_block_flag` of that sub-block is set to 1. For example, in... Figure 19 The example shows setting the `coded_sub_block_flag` value to 1 for the first, second, and third sub-blocks containing non-zero residual coefficients. When no non-zero residual coefficients exist within a 4x4 sub-block, the `coded_sub_block_flag` value is set to 0. For example, in... Figure 19 The text shows that the value of coded_sub_block_flag for the fourth sub-block, which does not include non-zero residual coefficients, is set to 0.

[0265] `rqt_root_cbf` and `coded_sub_block_flag` can be encoded in a hierarchical manner and can be signaled. For example, when `rqt_root_cbf` is 0, encoding / decoding `coded_sub_block_flag` is omitted. Conversely, when `rqt_root_cbf` is 1, and the current block size is larger than the sub-block size, `coded_sub_block_flag` is encoded and signaled for each sub-block within the current block.

[0266] Even residual coefficient encoding indicators between transform blocks and coded blocks can be encoded hierarchically and transmitted via signals. For example, a first residual coefficient encoding indicator indicating whether at least one transform block with non-zero residual coefficients is included among multiple transform blocks can be encoded / decoded. Depending on the value of the first residual coefficient encoding indicator, it can be determined whether a second residual coefficient encoding indicator is encoded / decoded for each transform block. In this document, at least one of the size and shape of a parent block comprising multiple transform blocks can have a predefined value, or can be determined by information transmitted via signals from a bitstream. Alternatively, at least one of the size and shape of a parent block can be determined based on the partitioning type of the coded tree block. For example, a square coded block or a non-square coded block comprising multiple non-square transform blocks can be defined as a parent block for multiple non-square transform blocks. Residual coefficient encoding indicators can be encoded hierarchically through two or more layers.

[0267] As mentioned above, the method of encoding residual coefficient coding indicators in a hierarchical manner can be called the hierarchical residual coefficient coding indicator (hierarchical coding block flag, HCBF) derivation method.

[0268] As a result of performing at least one of quadtree partitioning, binary tree partitioning, or ternary tree partitioning, a coded block with a width or height, or both, smaller than a predefined value can be generated. For example, a coded block with a width or height, or both, smaller than 4 can be generated. A coded block with a width or height, or both, smaller than 4 can be defined as an ultrasmall asymmetric coding unit.

[0269] The size, shape, or number of sub-blocks included in the current block can be determined based on at least one of the size of the current block, the shape of the current block, or the quantization parameters of the current block. For example, a sub-block included in a coded block with a width and height greater than 4 can be 4x4 in size. Therefore, the coded_sub_block_flag can be encoded for each 4x4 sub-block. Conversely, a sub-block included in a coded block with a width or height, or both, less than 4, can be 2x2, 2x4, 4x2, 2x8, or 8x2 in size. Therefore, the coded_sub_block_flag can be encoded for each sub-block of size 2x2, 2x4, 4x2, 2x8, or 8x2.

[0270] Figure 20 and Figure 21 This is a diagram illustrating the coding aspects of coded_sub_block_flag for ultra-small asymmetric coding units.

[0271] For a very small asymmetric coding block of size 2xN, CSBF can be encoded for each sub-block of size 2x2, 2x4, or 2x8.Figure 20 (a) is a diagram showing the case where the sub-block is 2x2 in size. Figure 20 (b) is a diagram showing the case where the sub-block is 2x4 in size. Figure 20 (c) is a diagram showing the case where the sub-block is 2x8 in size.

[0272] For an Nx2-sized ultra-small asymmetric coding block, CSBF can be encoded for each sub-block of size 2x2, 4x2, or 8x2. Figure 21 (a) is a diagram showing the case where the sub-block is 8x2 in size. Figure 21 (b) is a diagram showing the case where the sub-block is 4x2 in size. Figure 21 (c) is a diagram showing the case where the sub-block is 2x2 in size.

[0273] Information indicating at least one of the size, shape, or number of sub-blocks included in a coded block can be transmitted via a bitstream using signals. Based on this information, the size and / or shape of the sub-blocks included in a very small asymmetric coded block can be determined. CSBF can be encoded based on the sub-blocks determined using this information. This information can be transmitted via signals at the sequence header, picture header, slice header, tile group header, tile header, or block level. This information can indicate at least one of the size, shape, or number of sub-blocks included in a coded block. Alternatively, this information can indicate the smallest unit for encoding a CSBF, the largest unit for encoding a CSBF, or the difference between the smallest and largest units for encoding a CSBF.

[0274] Alternatively, sub-blocks can be defined based on a preset number of samples. Here, the preset number can be 4, 8, 16, or more. The preset number can be predefined in the encoder and decoder. Alternatively, the preset number can be determined based on at least one of the size of the coded block, the shape of the coded block, or the quantization parameters of the coded block. Alternatively, information for determining the preset number can be transmitted via a bitstream signal. This information can indicate the minimum number of samples for the sub-block encoding CSBF, the maximum number of samples for the sub-block encoding CSBF, or the difference between the minimum and maximum number of samples.

[0275] Figure 22 This is a diagram illustrating an example of encoding residual coefficient encoding indicators for each sub-block containing a preset number of samples.

[0276] exist Figure 22 In the example shown, CU0 and CU2 are 1x16-sized ultrasmall asymmetric coding blocks, and CU1 is a 2x16-sized ultrasmall asymmetric coding block.

[0277] Assuming a group of 16 samples includes a CSBF, for CU0 and CU2, which each contain 16 samples, the CSBF can be encoded without dividing it into sub-blocks. Alternatively, for CU0 and CU2, encoding / decoding the CSBF can be omitted, and the rqt_root_flag can be encoded / decoded.

[0278] The CU1, comprising 32 samples, can be divided into two sub-blocks, and CSBF can be encoded / decoded for each sub-block. Figure 20 The example shown depicts a 2x8 sub-block within CU1, but the size of the sub-blocks can be set differently. For instance, CU1 could be divided into two 1x16 sub-blocks.

[0279] When a non-zero residual coefficient is included in the current block, the absolute value and / or sign of each residual coefficient is encoded / decoded according to the scan order of the current block (S1840).

[0280] When at least one non-zero residual coefficient is included in the current block or sub-block, the residual coefficients are encoded / decoded using a residual coefficient level indicator that indicates whether the residual coefficient is zero. The residual coefficient level indicator is a 1-bit flag (e.g., "significant_flag" or "sig_coeff_flag") and indicates whether each residual coefficient within the current block is zero. For example, a significant_flag value of 1 indicates that the residual coefficient is not zero, while a significant_flag value of 0 indicates that the residual coefficient is zero. Non-zero residual coefficients can be referred to as valid residual coefficients.

[0281] The transform coefficient levels of all residual coefficients within a block can be referred to as a residual coefficient level map (significant map). The encoder can encode the residual coefficient level map and then encode the absolute values ​​and signs of the significant residual coefficients. Specifically, the encoder can encode the position of the last significant transform coefficient according to the scan order of the current block, and can perform an inverse transform scan starting from the last significant transform coefficient to encode the residual coefficient level indicator for each residual coefficient. An inverse transform scan can be performed in the reverse direction of the scan order. Additionally, the absolute values ​​and signs of the significant transform coefficients can be encoded.

[0282] The absolute value of a valid transform coefficient can be encoded using at least one of the following: a flag indicating whether the absolute value of a valid transform coefficient is greater than 1 (gt1_flag), a flag indicating whether the absolute value of a valid transform coefficient or the value obtained by subtracting 1 from a valid transform coefficient is even (parity_level_flag), a flag indicating whether the absolute value of a valid transform coefficient is greater than 3 (gt3_flag), and the value obtained by subtracting 3 from the absolute value of a valid transform coefficient (abs_remainder).

[0283] The encoding of CSBF, the encoding of residual coefficient level indicators, and the encoding of the absolute value and sign of effective transform coefficients can be repeated until the encoding of the last residual coefficients is completed.

[0284] The decoder can decode the position of the last valid residual coefficient and can decode the valid residual coefficient level indicator of the sample from the first sample to the last sample.

[0285] When the absolute value of the indicated valid transform coefficient is not greater than 1, the absolute value of the valid transform coefficient is determined to be 1. When the absolute value of the indicated valid transform coefficient is greater than 1 and not greater than 3, the absolute value of the valid transform coefficient is determined by a flag indicating whether the absolute value of the valid transform coefficient is even. For example, when the value obtained by subtracting 1 from the valid transform coefficient is odd, the absolute value of the valid transform coefficient is determined to be 2. When the value obtained by subtracting 1 from the valid transform coefficient is even, the absolute value of the valid transform coefficient is determined to be 3. When the absolute value of the valid transform coefficient is greater than 3, the absolute value of the valid transform coefficient is determined by decoding the value obtained by subtracting 3 from the absolute value of the valid transform coefficient and adding 3 to the decoded value.

[0286] Based on the residual coefficients at a preset position included in a block of size MxN, the values ​​of the residual coefficients along a preset direction from the preset transform coefficients can be adjusted. Here, the block of size MxN can refer to a transform block or a sub-block included in a transform block. The values ​​of M and N can be determined based on the size of the transform block or sub-block. The preset direction can be determined based on the scan order of the current block. For example, the residual coefficient values ​​of residual coefficients whose scan order has a lower priority than the scan order of the residual coefficients at the preset position, or the residual coefficient values ​​of residual coefficients whose scan order has a higher priority than the scan order of the residual coefficients at the preset position, can be adjusted. Alternatively, the preset direction can be a direction predefined in the encoder and decoder. For example, the preset direction can be a leftward direction, a rightward direction, an upward direction, a downward direction, or a diagonal direction.

[0287] Adjusting the residual coefficient can refer to changing the value of the residual coefficient to be the same as the residual coefficient adjustment reference value, changing the value of the residual coefficient to be less than the residual coefficient adjustment reference value, or changing the value of the residual coefficient to 0.

[0288] Figure 23 This is a graph showing an example of adjusting the values ​​of the residual coefficients.

[0289] The scan order of each of the MxN residual coefficients included in an MxN block can be identified by the scan index of each residual coefficient. Additionally, a preset scan order can be defined as the residual coefficient adjustment index. The absolute value of a residual coefficient whose scan index has a lower priority than the residual coefficient adjustment index can be adjusted to a residual coefficient adjustment reference value or smaller.

[0290] For example, the absolute value of the residual coefficients that are later than the eighth in the scan sequence can be set to not exceed 1. (See reference...) Figure 23 Under diagonal scanning, the residual coefficients from scan index 0 to scan index 7 are (13, 10, 2, 4, -5, 8, -1, -3, 1). Among the residual coefficients that are later in the scan order than the eighth (i.e., residual coefficients with scan indices greater than 7), the absolute values ​​of residual coefficients with an absolute value greater than 1 can be set to 1 or less. For example, the absolute value of the fourteenth residual coefficient (i.e., the residual coefficient with scan index 13) is 2, and therefore the absolute value of the residual coefficient can be changed to 1.

[0291] The adjustment index can be determined based on the number of samples included in the block. For example, the adjustment index for M x N residual coefficients can be set to (M x N) >> 1. Alternatively, the adjustment index can be determined based on the position of the last valid transform coefficient. For example, the adjustment index can be set to (LastSigPos+1) >> 1 or (LastSigPos-1) >> 1. The position of the last valid transform coefficient can be represented by LastSigPos. Alternatively, information for determining the adjustment index can be sent via a bitstream using signals. This information can be sent via sequence parameters, picture levels, title sequences, tile groups, tiles, transform blocks, or residual coding syntax.

[0292] Alternatively, the value of the adjustment index can be predefined in the encoder and decoder.

[0293] The residual coefficient adjustment reference value can be set to a non-zero integer value. The residual coefficient adjustment reference value can be predefined in the encoder or decoder. Alternatively, the residual coefficient adjustment reference value can be defined at the sequence level, image level, slice level, or block level. For example, a predefined residual coefficient adjustment value can be used in all slices, or a residual coefficient adjustment value can be determined for each slice.

[0294] Alternatively, based on the adjustment index, a residual coefficient adjustment reference value can be determined. For example, the following can be set as the residual coefficient adjustment reference value: the value of the residual coefficient of the scan index being the adjustment index, the value of the first effective residual coefficient among the residual coefficients of the scan index being greater than the adjustment index, or the value of the first effective residual coefficient among the residual coefficients of the scan index being less than the adjustment index.

[0295] The absolute value of valid residual coefficients with a lower priority than the adjustment index in the scan index can be set to the residual coefficient adjustment reference value or less. Therefore, the absolute value of the residual coefficients may not need to be encoded, or only the minimum information used to determine the absolute value of the residual coefficients may need to be encoded. For example, when the residual coefficient adjustment reference value is 1, the absolute value of valid residual coefficients with a lower priority than the adjustment index in the scan index is determined to be 1. As a result, the absolute value of residual coefficients with a lower priority than the adjustment index in the scan index can be set to the same value as the residual coefficient level indicator. Therefore, encoding / decoding of at least one of gt1_flag, parity_level_flag, gt3_flag, and / or abs_remainder used to determine the absolute value of valid residual coefficients can be omitted.

[0296] For example, when the residual coefficient adjustment reference value is 2, the absolute value of the effective residual coefficients that determine the scan index has a lower priority than the adjustment index is 1 or 2. As a result, the absolute value of the residual coefficients that have a lower priority than the adjustment index can be set to be the same as the sum of the values ​​of the residual coefficient level indicator and gt1_flag. Therefore, the encoding / decoding of at least one of parity_level_flag, gt3_flag, and abs_remainder used to determine the absolute value of the effective residual coefficients can be omitted.

[0297] For example, when the residual coefficient adjustment reference value is 3, the absolute value of the effective residual coefficients that determine the scan index has a lower priority than the adjustment index is 1, 2, or 3. As a result, the absolute value of the residual coefficients that have a lower priority than the adjustment index can be set to the same as the sum of the residual coefficient level indicator, the value of gt1_flag, and parity_level_flag. Therefore, the encoding / decoding of at least one of gt3_flag and abs_remainder used to determine the absolute value of the effective residual coefficients can be omitted.

[0298] In other words, by adjusting the absolute value of the effective residual coefficients to a residual coefficient adjustment reference value or smaller, the number of bits required to determine the absolute value of the effective residual coefficients is reduced, thereby improving encoding / decoding efficiency.

[0299] Information indicating whether the absolute value of the residual coefficients has been adjusted can be sent via a bitstream using signals. This information can be a 1-bit flag. For example, a true flag indicates that the absolute value of the residual coefficients within the current block where the scan index is greater than the adjustment index has been adjusted. A false flag indicates that the absolute value of the residual coefficients within the current block has not been adjusted.

[0300] Figure 24 This is a flowchart illustrating the process of obtaining residual samples according to an embodiment of the present invention.

[0301] First, the residual coefficients of the current block can be obtained (S2410). The decoder can obtain the residual coefficients through a coefficient scan method. For example, the decoder can perform a coefficient scan using a diagonal scan, a jig-zag scan, an up-right scan, a vertical scan, or a horizontal scan, and can obtain the residual coefficients in the form of a two-dimensional block.

[0302] Inverse quantization can be performed on the residual coefficients of the current block (S2420).

[0303] It can be determined whether to skip the inverse transform of the dequantized residual coefficients of the current block (S2430). Specifically, the decoder can determine whether to skip the inverse transform on at least one of the horizontal or vertical directions of the current block. When it is determined that the inverse transform on at least one of the horizontal or vertical directions of the current block is to be applied, the residual sample of the current block can be obtained by performing the inverse transform on the dequantized residual coefficients of the current block (S2440). Here, at least one of DCT, DST, or KLT can be used to perform the inverse transform.

[0304] When the inverse transform is skipped in both the horizontal and vertical directions of the current block, the inverse transform is not performed in the horizontal and vertical directions of the current block. In this case, the residual sample of the current block can be obtained by scaling the inverse quantization residual coefficients with a predetermined value (S2450).

[0305] Skipping the inverse transform in the horizontal direction means performing the inverse transform in the vertical direction instead of the horizontal one. In this case, scaling can be performed in the horizontal direction.

[0306] Skipping the inverse transform in the vertical direction means performing the inverse transform in the horizontal direction instead of the vertical one. In this case, scaling can be performed in the vertical direction.

[0307] The suitability of the inverse transform skipping technique for the current block can be determined based on the type of partitioning. For example, if the current block is generated via a binary tree-based partition, the inverse transform skipping scheme can be restricted to this block. Therefore, when the current block is generated via a binary tree-based partition, the residual sample of the current block can be obtained by performing an inverse transform on the current block. Furthermore, when the current block is generated via a binary tree-based partition, the encoding / decoding of information indicating whether to skip the inverse transform (e.g., transform_skip_flag) can be omitted.

[0308] Alternatively, when generating the current block through binary tree-based partitioning, the inverse transform skipping scheme can be restricted to at least one of the horizontal or vertical directions. Here, the direction in which the inverse transform skipping scheme is restricted can be determined based on information decoded from the bitstream, or adaptively based on the size of the current block, the shape of the current block, or the intra-frame prediction mode of the current block.

[0309] For example, when the current block is a non-square block with a width greater than its height, inverse transformation skipping schemes can be allowed only in the vertical direction, while inverse transformation skipping schemes can be restricted in the horizontal direction. That is, when the current block is 2NxN, the inverse transformation is performed in the horizontal direction of the current block, and the inverse transformation can be selectively performed in the vertical direction.

[0310] On the other hand, when the current block is a non-square block with a height greater than its width, inverse transformation skipping schemes can be allowed only in the horizontal direction, while inverse transformation skipping schemes can be restricted in the vertical direction. That is, when the current block is Nx2N, the inverse transformation is performed in the vertical direction of the current block, and the inverse transformation can be selectively performed in the horizontal direction.

[0311] Compared to the example above, when the current block is a non-square block with a width greater than its height, the inverse transformation skipping scheme can be allowed only in the horizontal direction, and when the current block is a non-square block with a height greater than its width, the inverse transformation skipping scheme can be allowed only in the vertical direction.

[0312] Information indicating whether to skip the inverse transform relative to the horizontal direction or relative to the vertical direction can be transmitted via a bitstream signal. For example, the information indicating whether to skip the inverse transform in the horizontal direction is a 1-bit flag 'hor_transform_skip_flag', and the information indicating whether to skip the inverse transform in the vertical direction is a 1-bit flag 'ver_transform_skip_flag'. The encoder can encode at least one of 'hor_transform_skip_flag' or 'ver_transform_skip_flag' based on the shape of the current block. Furthermore, the decoder can determine whether to skip the inverse transform in the horizontal or vertical direction by using at least one of 'hor_transform_skip_flag' or 'ver_transform_skip_flag'.

[0313] It can be configured to skip the inverse transform of the current block in either direction based on the partitioning type of the current block. For example, if the current block is generated by a binary tree-based partition, the inverse transform in either the horizontal or vertical direction can be skipped. That is, if the current block is generated by a binary tree-based partition, it can be determined to skip the inverse transform of the current block in at least one of the horizontal or vertical directions without requiring encoding / decoding information indicating whether to skip the inverse transform of the current block (e.g., transform_skip_flag, hor_transform_skip_flag, ver_transform_skip_flag).

[0314] This disclosure can also have the following configurations:

[0315] 1. A method for decoding an image, the method comprising:

[0316] Determine whether non-zero residual coefficients are included in the current block; and

[0317] When it is determined that the current block contains non-zero residual coefficients, the residual coefficients included in the current block are decoded according to the scan order of the current block.

[0318] Among them, the absolute value of the residual coefficient of the scan index is equal to or less than the residual coefficient adjustment reference value.

[0319] 2. The method according to Scheme 1, wherein the residual coefficient adjustment index is determined based on at least one of the size of the current block and the position of the last effective residual coefficient.

[0320] 3. According to the method of Scheme 1, wherein the residual coefficient adjustment reference value is 1, the absolute value of the residual coefficient is set to be the same as the value of the residual level indicator, and the residual level indicator indicates whether the absolute value of the residual coefficient is greater than 0.

[0321] 4. The method according to Scheme 1, wherein the residual coefficient adjustment reference value is set differently for each slice.

[0322] 5. The method according to Scheme 1, wherein the residual coefficient adjustment reference value is set to be the same as the absolute value of the residual coefficient of the scan index being the residual coefficient adjustment index.

[0323] 6. The method according to Scheme 1, wherein the determination of whether the current block includes non-zero residual coefficients is performed based on the residual coefficient encoding indicator of the current block.

[0324] 7. A method for encoding an image, the method comprising:

[0325] Determine whether non-zero residual coefficients are included in the current block; and

[0326] When it is determined that the current block contains non-zero residual coefficients, the residual coefficients included in the current block are encoded according to the scan order of the current block.

[0327] Among them, the absolute value of the residual coefficient of the scan index is equal to or less than the residual coefficient adjustment reference value.

[0328] 8. The method according to Scheme 7, wherein the residual coefficient adjustment index is determined based on at least one of the size of the current block and the position of the last valid residual coefficient.

[0329] 9. The method according to Scheme 7 further includes:

[0330] Encode the residual level indicator that indicates whether the absolute value of the residual coefficient is greater than 0.

[0331] The residual coefficient adjustment reference value is 1, and the absolute value of the residual coefficient is set to be the same as the value of the residual level indicator.

[0332] 10. The method according to Scheme 7, wherein the residual coefficient adjustment reference value is set differently for each slice.

[0333] 11. The method according to Scheme 7, wherein the residual coefficient adjustment reference value is set to be the same as the absolute value of the residual coefficient of the scan index being the residual coefficient adjustment index.

[0334] 12. The method according to Scheme 7 further includes:

[0335] The residual coefficient encoding indicator of the current block is encoded, and the residual coefficient encoding indicator indicates whether the current block includes non-zero residual coefficients.

[0336] 13. An apparatus for decoding an image, the apparatus comprising:

[0337] The decoding module determines whether the current block contains non-zero residual coefficients, and when it is determined that the current block contains non-zero residual coefficients, it decodes the residual coefficients included in the current block according to the scan order of the current block.

[0338] Among them, the absolute value of the residual coefficient of the scan index is equal to or less than the residual coefficient adjustment reference value.

[0339] 14. An apparatus for encoding an image, the apparatus comprising:

[0340] An encoding module determines whether a non-zero residual coefficient is included in the current block, and when it is determined that a non-zero residual coefficient is included in the current block, encodes the residual coefficients included in the current block according to the scan order of the current block.

[0341] Among them, the absolute value of the residual coefficient of the scan index is equal to or less than the residual coefficient adjustment reference value.

[0342] The names of the syntax used in the above embodiments are given only for ease of description.

[0343] Application of the embodiments described in relation to decoding or encoding processes to encoding or decoding processes is included within the scope of this invention. Changing the order of embodiments described in a predetermined order to a different order is also included within the scope of this invention.

[0344] Although the above embodiments have been described based on a series of steps or flowcharts, they do not limit the temporal order of the invention and can be executed simultaneously or in different orders as needed. Furthermore, each of the components (e.g., units, modules, etc.) constituting the block diagrams in the above embodiments can be implemented by hardware devices or software and multiple components. Alternatively, multiple components can be combined and implemented by a single hardware device or software. The above embodiments can be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. This computer-readable recording medium can include one or a combination of program commands, data files, data structures, etc. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy optical discs; media; and hardware devices (e.g., ROMs, RAMs, flash memory, etc.) specifically configured to store and execute program instructions. The hardware device can be configured to operate as one or more software modules to perform the processing according to the invention, and vice versa.

[0345] Industrial applicability

[0346] This invention can be applied to electronic devices capable of encoding / decoding video.

Claims

1. A method of decoding an image, the method comprising: determining a current transform block by partitioning a coding tree block based on a tree-based partitioning, the tree-based partitioning comprising at least one of a quad-tree partitioning and a binary-tree partitioning; determining whether at least one non-zero residual coefficient is included in the current transform block; and in response to determining that at least one non-zero residual coefficient is included in the current transform block, obtaining residual coefficients of the current transform block according to a scan order of the current transform block, wherein the residual coefficients of the current transform block are obtained based on coefficient level information of the current transform block, wherein the coefficient level information comprises at least one of a first residual level flag, a parity flag and a second residual level flag, wherein an absolute value of a residual coefficient is determined by: determining whether the absolute value of the residual coefficient is greater than 1 based on the first residual level flag, in response to the first residual level flag indicating that the absolute value of the residual coefficient is greater than 1, determining whether the absolute value of the residual coefficient is even or odd based on the parity flag, and determining whether the absolute value of the residual coefficient is greater than 3 based on the second residual level flag, wherein the parity flag and the second residual level flag are signaled via a bitstream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1, and wherein in response to the second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3, the absolute value of the residual coefficient is determined to be 2 or 3 according to a value of the parity flag. Each residual coefficient of residual coefficients earlier than a reference position at a predefined location in a decoding device in the scan order is equal to 0 in case information explicitly signaled via a bitstream has a value other than 0.

2. The method of claim 1, wherein, Coefficient level information of each residual coefficient of residual coefficients earlier than the reference position in the scan order is not signaled via a bitstream.

3. The method of claim 2, wherein, 4. The method of claim 1, further comprising: determining whether to skip inverse transform of the current transform block based on a transform skip flag indicating whether to skip inverse transform, wherein the transform skip flag is not signaled via a bitstream and inverse transform of the current transform block is not allowed to be skipped when the current transform block is one of two transform blocks generated by partitioning a coding block.

5. A method of encoding an image, the method comprising: determining a current transform block by partitioning a coding tree block based on a tree-based partitioning, the tree-based partitioning comprising at least one of a quad-tree partitioning and a binary-tree partitioning; determining whether at least one non-zero residual coefficient is included in the current transform block; and in response to determining that at least one non-zero residual coefficient is included in the current transform block, encoding residual coefficients of the current transform block according to a scan order of the current transform block to generate coefficient level information of the current transform block, wherein the coefficient level information comprises at least one of a first residual level flag, a parity flag and a second residual level flag, ​ ​ wherein a first residual level flag indicating whether an absolute value of the residual coefficient is greater than 1 is encoded into the bitstream, wherein the even-odd flag and a second residual level flag indicating whether the absolute value of the residual coefficient is greater than 3 are further encoded into the bitstream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1, and wherein the second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3 is encoded into the bitstream in response to that the absolute value of the residual coefficient is 2 or 3.

6. The method of claim 5, wherein, In a case where the information explicitly encoded into the bitstream is set to be equal to a value other than 0, each residual coefficient of the residual coefficients in the scan order earlier than a residual coefficient at a reference position predefined in the encoding apparatus is equal to 0.

7. The method of claim 6, wherein, Coefficient level information of each residual coefficient of the residual coefficients in the scan order earlier than the residual coefficient at the reference position is not encoded into the bitstream.

8. The method of claim 5, further comprising: determining whether to skip a transform of the current transform block, wherein a transform skip flag indicating whether to skip a transform is not encoded into the bitstream and a transform of the current transform block is not allowed to be skipped in a case where the current transform block is one of two transform blocks generated by partitioning a coding block.

9. A method of transmitting image data, comprising: obtaining a bitstream of the image data, wherein the bitstream is generated by determining a current transform block by partitioning a coding tree block based on a tree-based partitioning including at least one of a quad-tree partitioning and a binary-tree partitioning, determining whether at least one non-zero residual coefficient is included in the current transform block, and in response to determining that the at least one non-zero residual coefficient is included in the current transform block, encoding residual coefficients of the current transform block according to a scan order of the current transform block; and transmitting the bitstream, wherein the bitstream includes coefficient level information generated by encoding the residual coefficients of the current transform block, wherein the coefficient level information includes at least one of a first residual level flag, an even-odd flag, and a second residual level flag, wherein the first residual level flag indicates whether an absolute value of the residual coefficient is greater than 1, the even-odd flag indicates whether the absolute value of the residual coefficient is even or odd, and the second residual level flag indicates whether the absolute value of the residual coefficient is greater than 3, wherein the even-odd flag and the second residual level flag are encoded into the bitstream only when the first residual level flag indicates that the absolute value of the residual coefficient is greater than 1, and wherein the second residual level flag indicating that the absolute value of the residual coefficient is not greater than 3 is encoded into the bitstream in response to that the absolute value of the residual coefficient is 2 or 3.

Images