Image encoding / decoding method and device thereof

Abstract

Disclosed is an image encoding / decoding method and device thereof. Specifically, the image encoding method may include the following steps: generating a quantized transform block by performing transform and quantization on a residual signal of a current block; dividing the quantized transform block into a plurality of coefficient groups; determining a first scanning order representing a scanning order among coefficients of the coefficient groups; and entropy encoding the coefficients of the quantized transform block according to the first scanning order and a second scanning order representing a scanning order among the plurality of coefficient groups.

Description

[0001] This application is a divisional application of a patent application with application number 201880031909.X (PCT / KR2018 / 002952) filed on November 14, 2019, with an international application date of March 13, 2018, and the invention name being “Image encoding / decoding method and device thereof”. Technical Field

[0002] The present disclosure relates to a still image or moving image processing method, and more particularly, to a method for encoding / decoding a transformed residual signal and a device supporting the same. Background Art

[0003] Compression coding refers to a series of signal processing techniques for sending digitized information via communication lines, or a technology for storing information in a form suitable for storage media. Media including pictures, images, audio, etc. can be the target for compression coding, and specifically, the technology of performing compression coding on pictures is called video image compression.

[0004] The next generation of video content is assumed to be characterized by high spatial resolution, high frame rate, and high dimensionality of scene representation. To process such content, it will lead to a dramatic increase in memory storage, memory access rate, and processing power.

[0005] Therefore, there is a need to design a coding tool that can efficiently process the next generation video content. Summary of the invention

[0006] Technical issues

[0007] The purpose of the present disclosure is to propose a method for encoding / decoding a transformed residual signal.

[0008] Furthermore, an object of the present disclosure is to propose a method for configuring a scanning order among partition coefficient groups from a transform block in order to perform entropy coding on a residual signal.

[0009] Furthermore, an object of the present disclosure is to propose a method for configuring a scanning order among coefficients in a partitioned coefficient group from a transform block in order to perform entropy coding on a residual signal.

[0010] The technical objectives of the present disclosure are not limited to the above-mentioned technical objectives, and a person of ordinary skill in the art will clearly recognize other technical objectives not mentioned above from the following description.

[0011] Technical Solution

[0012] In one aspect of the present disclosure, an image encoding method may include: generating a quantized transform block by performing transform and quantization on a residual signal of a current block; dividing the quantized transform block into a plurality of coefficient groups; determining a first scanning order representing a scanning order among coefficients of the coefficient groups; and entropy encoding the coefficients of the quantized transform block according to the first scanning order and a second scanning order representing a scanning order among the plurality of coefficient groups.

[0013] Preferably, determining the first scanning order may include: determining distance values ​​among respective coefficients of the coefficient group based on an upper left coefficient of the coefficient group; and assigning scanning indices to coefficients having the same distance value according to a predetermined order while sequentially increasing the distance value.

[0014] Preferably, determining the distance value among each coefficient of the coefficient group may include: when the coefficient group is a non-square block, determining a horizontal increment representing the increase in the coordinate value in the horizontal direction of each coefficient and a vertical increment representing the increase in the coordinate value in the vertical direction based on the ratio of the width and height of the coefficient group, and the distance value of each coefficient may be determined by a value obtained by summing the horizontal increment and the vertical increment.

[0015] Preferably, when the width of the coefficient group is greater than the height, the horizontal increment may be determined as 1 and the vertical increment may be determined as 2, and when the height of the coefficient group is greater than the width, the horizontal increment may be determined as 2 and the vertical increment may be determined as 1.

[0016] Preferably, partitioning the quantized transform block into a plurality of coefficient groups may include: partitioning the quantized transform block hierarchically, and partitioning the partitioned quantized transform block into coefficient groups with lower depths, and the second scanning order may include a scanning order among coefficient groups with respective depths from the quantized transform block.

[0017] Preferably, when the quantized transform block is a non-square block, the quantized transform block may be divided into coefficient groups consisting of a specific number of coefficients, and the specific number may be determined according to a ratio of a width and a height of the quantized transform block.

[0018] In another aspect of the present disclosure, a method for decoding an image may include: dividing a current processing block into multiple coefficient groups; determining a first scanning order representing a scanning order among coefficients in the coefficient groups; entropy decoding a bit stream output from an encoder to generate quantized transform coefficients; and arranging the coefficients of the quantized transform block according to the first scanning order and a second scanning order representing a scanning order among the multiple coefficient groups to generate a quantized transform block of the current processing block.

[0019] In another aspect of the present disclosure, the step of determining the first scanning order may include: determining the distance values ​​among the respective coefficients of the coefficient group based on the upper left coefficient of the coefficient group; and assigning scanning indices to coefficients having the same distance values ​​according to a predetermined order while sequentially increasing the distance values.

[0020] Preferably, determining the distance values ​​among the individual coefficients of the coefficient group may include: when the coefficient group is a non-square block, determining a horizontal increment representing the amount of increase in the coordinate value in the horizontal direction of each coefficient and a vertical increment representing the amount of increase in the coordinate value in the vertical direction based on the ratio of the width and height of the coefficient group, and the distance value of each coefficient may be determined by the value obtained by summing the horizontal increment and the vertical increment.

[0021] Preferably, when the width of the coefficient group is greater than the height, the horizontal increment may be determined as 1 and the vertical increment may be determined as 2, and when the height of the coefficient group is greater than the width, the horizontal increment may be determined as 2 and the vertical increment may be determined as 1.

[0022] Preferably, splitting the current processing block into a plurality of coefficient groups may include: splitting the current processing block hierarchically; and splitting the split current processing block into coefficient groups with lower depths, and the second scanning order may include a scanning order among coefficient groups with respective depths from the current processing block.

[0023] Preferably, when the current processing block is a non-square block, the current processing block may be divided into coefficient groups consisting of a specific number of coefficients, and the specific number may be determined according to a ratio of a height to a width of the current processing block.

[0024] In yet another aspect of the present invention, an apparatus for decoding an image may include: a coefficient group splitting unit which splits a current processing block into a plurality of coefficient groups; and a scanning order determining unit which determines a first scanning order representing a scanning order among coefficients of the coefficient groups; a transform coefficient generating unit which entropy decodes a bit stream output from an encoder to generate quantized transform coefficients; and a transform block generating unit which arranges coefficients of the quantized transform block according to a second scanning order representing the first scanning order and a scanning order among a plurality of coefficient groups to generate a quantized transform block of the current processing block.

[0025] Beneficial Effects

[0026] According to an embodiment of the present disclosure, the amount of a residual signal can be saved when encoding a still image or a moving image.

[0027] Furthermore, according to an embodiment of the present disclosure, the amount of residual signal is saved to enhance encoding / decoding efficiency.

[0028] Furthermore, according to an embodiment of the present disclosure, a scanning order of transform coefficients is effectively configured to reduce the amount of data required to code the transform coefficients.

[0029] In addition, according to an embodiment of the present disclosure, a scanning order of transform coefficients is configured by considering a non-square block shape to reduce the amount of data required to code the transform coefficients and enhance compression performance.

[0030] Advantages obtainable in the present disclosure are not limited to the above-mentioned effects, and other unmentioned advantages will be clearly understood by those skilled in the art from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are included as a part of the specification to help understand the present disclosure, provide embodiments of the present disclosure, and describe the technical features of the present disclosure through the following description.

[0032] Figure 1 The diagram is a schematic block diagram of an encoder as an embodiment to which the present disclosure is applied, in which encoding of a still image or a video signal is performed.

[0033] Figure 2 The diagram is a schematic block diagram of a decoder to which an embodiment of the present disclosure is applied, in which decoding of a still image or a video signal is performed.

[0034] Figure 3 is a diagram for describing a partition structure of a coding unit that can be applied to the present disclosure.

[0035] Figure 4 is a diagram for describing a prediction unit that can be applied to the present disclosure.

[0036] Figure 5 A method for encoding a residual signal is illustrated as an embodiment to which the present disclosure is applied.

[0037] Figure 6 A method for decoding a residual signal as an embodiment to which the present disclosure is applied is illustrated.

[0038] Figure 7 is a diagram illustrating a method for scanning transform coefficients in units of coefficient groups as an embodiment to which the present disclosure is applied.

[0039] Figure 8 is a diagram illustrating a scanning order of transform coefficients for each coefficient group as an embodiment to which the present disclosure is applied.

[0040] Fig. 9 and Fig.10 is a diagram for describing a method for determining a partition structure of a transform block and a scanning order of transform coefficients as an embodiment to which the present disclosure is applied.

[0041] Fig.11 is a diagram illustrating one example of the positions of coefficients in a representation block as an embodiment to which the present disclosure is applied.

[0042] Fig.12 and Fig.13 is a diagram illustrating a scanning order depending on positions of transform coefficients in a block according to an embodiment of the present disclosure.

[0043] Fig.14 and Fig.15 is a diagram for describing a method for determining a scanning order of coefficients of a non-square block as an embodiment to which the present disclosure is applied.

[0044] Fig.16 and Fig.17 is a diagram for describing a method for determining a scanning order of coefficients of a non-square block in units of superpixels as an embodiment to which the present disclosure is applied.

[0045] Fig.18 is a diagram illustrating a method for determining a scanning order of coefficient groups as an embodiment to which the present disclosure is applied.

[0046] Fig.19 A method for encoding an image according to an embodiment of the present disclosure is illustrated.

[0047] Fig. 20 A method for decoding an image according to an embodiment of the present disclosure is illustrated.

[0048] Fig.21 A device for decoding an image according to an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0049] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. The description that will be described below in conjunction with the accompanying drawings is for describing exemplary embodiments of the present disclosure, and is not intended to describe the only embodiment that can implement the present disclosure. The following description includes specific details in order to provide a complete understanding of the present disclosure. However, it should be understood that for those skilled in the art, the present disclosure can be implemented without the specific details.

[0050] In some cases, in order to prevent the technical concepts of the present disclosure from being unclear, well-known structures or devices may be omitted, or the block diagram may be depicted centering on the core functions of the structure of the device.

[0051] In addition, although the currently widely used general terms are selected as terms as much as possible in the present disclosure, terms arbitrarily selected by the applicant are used in specific cases. Since the meaning of the terms in this case will be clearly described in the corresponding part of the specification, it should be understood that the present disclosure will not be simply interpreted by the terms used only in the description of the present disclosure, but the meaning of the terms should be found out.

[0052] Specific terms used in the following description may be provided to help understand the present disclosure. In addition, within the scope of the technical concept of the present disclosure, specific terms may be modified into other forms. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately replaced and interpreted in each compilation process.

[0053] Hereinafter, in the present disclosure, a “block” or a “unit” may mean a unit in which an encoding / decoding process such as prediction, transform, and / or quantization is performed and samples may be configured in a multi-dimensional array of samples (or pixels).

[0054] "Block" or "unit" may refer to a multidimensional array of samples of a luminance component, and may refer to a multidimensional array of samples of a chrominance component. In addition, a block or unit may be collectively referred to as a multidimensional array of samples of a luminance component and a multidimensional array of samples of a chrominance component.

[0055] For example, "block" or "unit" may be interpreted as including the following meanings, namely, all coding blocks (CBs), meaning an array of samples to be encoded / decoded; a coding tree block (CTB), consisting of a plurality of coding blocks; a prediction block (PB) (or prediction unit (PU)), meaning an array of samples to which the same prediction is applied; and a transform block (TB) (or transform unit (TU)), meaning an array of samples to which the same transform is applied.

[0056] In addition, unless otherwise mentioned in the present disclosure, "block" or "unit" may be interpreted as including the meaning of a syntax structure used during a process of encoding / decoding an array of samples of a luminance component and / or a chrominance component. Here, a syntax structure means a syntax element of 0 or more elements present in a bitstream in a specific order, and a syntax element means a data element represented in a bitstream.

[0057] For example, "block" or "unit" may be interpreted as including the following meanings, namely, all coding units (CUs), including coding blocks (CBs) and syntax structures used to encode coding blocks (CBs); coding tree units (CUs), consisting of multiple coding units; prediction units (PUs), including prediction blocks (PBs) and syntax structures used to predict prediction blocks (PBs); and transform units (TUs), including transform blocks (TBs) and syntax structures used to transform transform blocks (TBs).

[0058] In addition, in the present disclosure, a "block" or "unit" is not particularly limited to an array of square or rectangular samples (pixels), and may mean an array of polygonal samples (or pixels) having three or more vertices. In this case, the "block" or "unit" may be referred to as a polygonal block or polygonal unit.

[0059] Figure 1 A schematic block diagram of an encoder that performs still image or video signal encoding as an embodiment to which the present disclosure is applied is illustrated.

[0060] refer to Figure 1 , the encoder 100 may include a video segmentation unit 110, a subtractor 115, a transform unit 120, a quantization unit 130, a dequantization unit 140, an inverse transform unit 150, a filtering unit 160, a decoded picture buffer (DPB) unit 170, a prediction unit 180, and an entropy encoding unit 190. In addition, the prediction unit 180 may include an inter-prediction unit 181 and an intra-prediction unit 182.

[0061] The video segmentation unit 110 segments an input video signal (or picture or frame) input to the encoder 100 into one or more blocks.

[0062] The subtractor 115 generates a residual signal (or residual block) by subtracting the prediction signal (or prediction block) output by the prediction unit 180 (ie, the inter prediction unit 181 or the intra prediction unit 182) from the input video signal. The generated residual signal (or residual block) is sent to the transform unit 120.

[0063] The transform unit 120 generates a transform coefficient by applying a transform scheme (e.g., discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), or Karnan-Loev transform (KLT)) to the residual signal (or residual block). In this case, the transform unit 120 may generate a transform coefficient by performing the transform using a prediction mode applied to the residual block and a transform scheme determined based on the size of the residual block.

[0064] The quantization unit 130 quantizes the transform coefficient and transmits it to the entropy encoding unit 190, and the entropy encoding unit 190 performs an entropy encoding operation of the quantized signal and outputs it as a bitstream.

[0065] Meanwhile, the quantized signal output by the quantization unit 130 may be used to generate a prediction signal. For example, a residual signal may be reconstructed by applying dequantization and inverse transformation to the quantized signal via the dequantization unit 140 and the inverse transformation unit 150. A reconstructed signal (or a reconstructed block) may be generated by adding the reconstructed residual signal to the prediction signal output by the inter prediction unit 181 or the intra prediction unit 182.

[0066] Meanwhile, during such a compression process, adjacent blocks are quantized by different quantization parameters. Therefore, artifacts showing the edge of the block may appear. Such a phenomenon is called block artifact, which is one of the important factors for evaluating image quality. In order to reduce such artifacts, a filtering process can be performed. Through such a filtering process, the block artifact is removed, and the error of the current picture is reduced at the same time, thereby improving the image quality.

[0067] The filtering unit 160 applies filtering to the reconstructed signal and outputs it via a playback device or sends it to the decoded picture buffer 170. The filtered signal sent to the decoded picture buffer 170 may be used as a reference picture in the inter-frame prediction unit 181. As described above, the coding rate and image quality may be improved by using the filtered picture as a reference picture in the inter-picture prediction mode.

[0068] The decoded picture buffer 170 may store the filtered picture so as to use it as a reference picture in the inter prediction unit 181 .

[0069] The inter-frame prediction unit 181 performs temporal prediction and / or spatial prediction with reference to the reconstructed picture to remove temporal redundancy and / or spatial redundancy. In this case, a block artifact or a ring artifact may occur because the reference picture used to perform the prediction is a transformed signal, which is quantized or dequantized in block units when it is previously encoded / decoded.

[0070] Here, since a reference picture used to perform prediction is a signal encoded / decoded in units of blocks at a previous time, a blocking artifact or a ringing artifact may exist.

[0071] Therefore, the inter-frame prediction unit 181 can interpolate signals between pixels in sub-pixel units by applying a low-pass filter to resolve performance degradation or enhance prediction of motion prediction due to discontinuity or quantization of such signals. Here, sub-pixels refer to virtual pixels generated by applying interpolation filters, and integer pixels refer to actual pixels present in the reconstructed picture. As interpolation methods, linear interpolation, bilinear interpolation, Wiener filter, etc. can be used.

[0072] An interpolation filter is applied to the reconstructed picture to improve the accuracy of prediction. For example, the inter-frame prediction unit 181 applies an interpolation filter to integer pixels to generate interpolation pixels, and can perform prediction by using an interpolation block composed of interpolation pixels.

[0073] The intra prediction unit 182 predicts the current block by referring to samples near the block to be currently encoded. The intra prediction unit 182 can perform the following processing to perform intra prediction. First, reference samples can be prepared, which are necessary for generating a prediction signal. In addition, a prediction signal (prediction block) can be generated by using the prepared reference samples. Thereafter, the prediction mode is encoded. In this case, the reference sample can be prepared by reference sample filling and / or reference sample filtering. Because the reference sample undergoes a prediction and reconstruction process, there may be a quantization error. Therefore, a reference sample filtering process can be performed for each prediction mode used for intra prediction to reduce such an error.

[0074] The prediction signal (or prediction block) generated by the inter prediction unit 181 or the intra prediction unit 182 may be used to generate a reconstructed signal (or reconstructed block) or to generate a residual signal (or residual block).

[0075] Figure 2 A schematic block diagram of a decoder that performs still image or video signal decoding as an embodiment to which the present disclosure is applied is illustrated.

[0076] refer to Figure 2 , the decoder 200 may include an entropy decoding unit 210, a dequantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a decoded picture buffer (DPB) 250, and a prediction unit 260. In addition, the prediction unit 260 may include an inter-prediction unit 261 and an intra-prediction unit 262.

[0077] Furthermore, the reconstructed video signal output via the decoder 200 may be played back via a playback device.

[0078] The decoder 200 may receive the Figure 1 The entropy decoding unit 210 performs an entropy decoding operation on the received signal.

[0079] The dequantization unit 220 obtains a transform coefficient from the entropy-decoded signal using quantization step size information.

[0080] The inverse transform unit 230 inversely transforms the transform coefficients by applying an inverse transform scheme to obtain a residual signal (or a residual block).

[0081] The adder 235 adds the acquired residual signal (or residual block) to the prediction signal (or prediction block) output from the prediction unit 260 (ie, the inter prediction unit 261 or the intra prediction unit 262) to generate a reconstructed signal (or reconstructed block).

[0082] The filtering unit 240 applies filtering to the reconstructed signal (or the reconstructed block) and outputs the filtered signal to the playback device, or transmits the filtered signal to the decoded picture buffer 250. The filtered signal transmitted to the decoded picture buffer 250 may be used as a reference picture in the inter-prediction unit 261.

[0083] In the present specification, the embodiments described in the filtering unit 160, the inter prediction unit 181, and the intra prediction unit 182 of the encoder 100 may be equally applied to the filtering unit 240, the inter prediction unit 261, and the intra prediction unit 262 of the decoder, respectively.

[0084] Block partitioning structure

[0085] Generally, a block-based image compression method is used in a compression technique (eg, HEVC) of a still image or a video. The block-based image compression method is a method of processing an image by dividing it into specific block units, and can reduce memory usage and calculation load.

[0086] Figure 3 is a diagram describing a partition structure of a coding unit that can be applied to the present disclosure.

[0087] The encoder divides a single image (or picture) into coding tree units (CTUs) in a grid form and encodes the CTUs one by one in a raster scan order.

[0088] In HEVC, the size of a CTU may be determined as one of 64×64, 32×32, and 16×16. The encoder may select and use the size of a CTU based on the resolution of an input video signal or the characteristics of an input video signal. A CTU includes a coding tree block (CTB) for a luma component and CTBs for two chroma components corresponding thereto.

[0089] One CTU can be split in a quadtree structure. That is, one CTU can be split into four units each having a square form and having half the horizontal size and half the vertical size, thereby generating a coding unit (CU). Such a split of the quadtree structure can be performed recursively. That is, a CU is hierarchically split from a CTU in a quadtree structure.

[0090] CU refers to a basic unit for a processing process (e.g., coding for performing intra / inter prediction) of an input video signal. A CU includes a coding block (CB) for a luma component and CBs for two chroma components corresponding to the luma component. In HEVC, the CU size may be determined as one of 64×64, 32×32, 16×16, and 8×8.

[0091] refer to Figure 3 , the root node of the quadtree is related to the CTU. The quadtree is partitioned until a leaf node is reached. The leaf node corresponds to the CU.

[0092] This is described in more detail. A CTU corresponds to a root node and has a minimum depth (ie, depth=0) value. Depending on the characteristics of an input video signal, a CTU may not be split. In this case, a CTU corresponds to a CU.

[0093] The CTU can be split in a quadtree form. Therefore, a lower node is generated, that is, a depth of 1 (depth=1). In addition, a node (i.e., a leaf node) that belongs to a lower node with a depth of 1 and is no longer split corresponds to a CU. For example, in Figure 3 In (B), CU(a), CU(b), and CU(j) corresponding to nodes a, b, and j have been split once from the CTU and have a depth of 1.

[0094] At least one of the nodes with depth 1 can be split in a quadtree form. Therefore, a lower node with depth 1 (ie, depth = 2) is generated. In addition, a node (ie, leaf node) that belongs to the lower node with depth 2 and is no longer split corresponds to a CU. For example, in Figure 3 In (B), CU(c), CU(h), and CU(i) corresponding to nodes c, h, and i have been split twice from the CTU and have a depth of 2.

[0095] In addition, at least one of the nodes with depth 2 may be split again in a quadtree form. Therefore, a lower node with depth 3 (ie, depth=3) is generated. In addition, a node (ie, leaf node) that belongs to the lower node with depth 3 and is no longer split corresponds to a CU. For example, in Figure 3 In (B), CU(d), CU(e), CU(f) and CU(g) corresponding to nodes d, e, f and g have been split three times from the CTU and have a depth of 3.

[0096] In the encoder, the maximum size or minimum size of a CU may be determined based on the characteristics of a video image (e.g., resolution) or by considering the coding rate. In addition, information about the maximum or minimum size or information that can obtain the information may be included in the bitstream. A CU with a maximum size is called a maximum coding unit (LCU), and a CU with a minimum size is called a minimum coding unit (SCU).

[0097] In addition, the CU with a tree structure can be hierarchically partitioned with a predetermined maximum depth information (or maximum level information). In addition, each partitioned CU can have depth information. Since the depth information indicates the partition count and / or the degree of the CU, it can include information about the CU size.

[0098] Since the LCU is partitioned in a quadtree shape, the size of the SCU may be obtained by using the size of the LCU and the maximum depth information. Or, conversely, the size of the LCU may be obtained by using the size of the SCU and the maximum depth information of the tree.

[0099] For a single CU, information indicating whether the corresponding CU is split (e.g., a split CU flag (split_cu_flag)) may be forwarded to the decoder. This split information is included in all CUs except the SCU. For example, when the value of the flag indicating whether the split is "1", the corresponding CU is further split into four CUs, and when the value of the flag indicating whether the split is "0", the corresponding CU is no longer split, and the processing for the corresponding CU may be performed.

[0100] As described above, a CU is a basic unit for performing coding of intra prediction or inter prediction. HEVC divides a CU into prediction units (PUs) to more efficiently code an input video signal.

[0101] PU is a basic unit for generating a prediction block, and even in a single CU, a prediction block can be generated in different ways in units of PUs. However, intra prediction and inter prediction are not used together for PUs belonging to a single CU, and PUs belonging to a single CU are coded by the same prediction method (ie, intra prediction or inter prediction).

[0102] The PU is not partitioned in a quadtree structure, but is partitioned once in a predetermined shape in a single CU. This will be described with reference to the following drawings.

[0103] Figure 4 is a diagram describing a prediction unit that can be applied to the present disclosure.

[0104] The PU is partitioned differently depending on whether an intra prediction mode or an inter prediction mode is used as a coding mode of a CU to which the PU belongs.

[0105] Figure 4 (A) illustrates a PU when intra prediction mode is used, and Figure 4 (B) illustrates a PU when the inter prediction mode is used.

[0106] refer to Figure 4 (A), assuming that the size of a single CU is 2N×2N (N=4, 8, 16, and 32), the single CU can be partitioned into two types (ie, 2N×2N or N×N).

[0107] In this case, if a single CU is partitioned into PUs of a 2N×2N shape, this means that only one PU exists in the single CU.

[0108] Meanwhile, if a single CU is partitioned into N×N shaped PUs, the single CU is partitioned into four PUs, and a different prediction block is generated for each PU unit. However, such PU partitioning can be performed only when the size of the CB for the luminance component of the CU is the minimum size (ie, when the CU is an SCU).

[0109] refer to Figure 4 (B) Assuming that the size of a single CU is 2N×2N (N=4, 8, 16, and 32), the single CU can be partitioned into eight PU types (i.e., 2N×2N, N×N, 2N×N, N×2N, nL×2N, nR×2N, 2N×nU, and 2N×nD).

[0110] Like in intra prediction, PU partitioning in an N×N shape may be performed only when the size of the CB for the luma component of the CU is the minimum size (ie, the case where the CU is an SCU).

[0111] Inter prediction supports PU partitioning in a 2N×N shape partitioned in a horizontal direction and in an N×2N shape partitioned in a vertical direction.

[0112] In addition, inter prediction supports PU partitions of nL×2N, nR×2N, 2N×nU, and 2N×nD shapes, which are asymmetric motion partitions (AMP). In this case, "n" refers to a 1 / 4 value of 2N. However, if the CU to which the PU belongs is a CU of the smallest size, AMP may not be used.

[0113] In order to efficiently encode the input video signal in a single CTU, the optimal partitioning structure of the coding unit (CU), prediction unit (PU), and transform unit (TU) can be determined based on the minimum rate distortion value through the following processing process. For example, regarding the optimal CU partitioning process in a 64×64 CTU, the rate distortion cost can be calculated through the partitioning process from a CU of 64×64 size to a CU of 8×8 size. The detailed process is as follows.

[0114] 1) An optimal partition structure of PUs and TUs that generates a minimum rate-distortion value is determined by performing inter / intra prediction, transform / quantization, dequantization / inverse transform, and entropy coding on a CU of 64×64 size.

[0115] 2) The optimal partition structure of PU and TU is determined to partition a 64×64 CU into four CUs of 32×32 size, and generate a minimum rate-distortion value for each 32×32 CU.

[0116] 3) The optimal partition structure of PU and TU is determined to further partition the 32×32 CU into four CUs of 16×16 size, and generate a minimum rate-distortion value for each 16×16 CU.

[0117] 4) The optimal partition structure of PU and TU is determined to further partition the 16×16 CU into four CUs of 8×8 size, and generate a minimum rate-distortion value for each 8×8 CU.

[0118] 5) Determine the optimal partition structure of the CU in the 16×16 block by comparing the rate distortion value of the 16×16 CU obtained in process 3) with the increase of the rate distortion values ​​of the four 8×8 CUs obtained in process 4). This process is also performed for the remaining three 16×16 CUs in the same manner.

[0119] 6) Determine the optimal partition structure of the CU in the 32×32 block by comparing the rate distortion value of the 32×32 CU obtained in process 2) with the increase of the rate distortion values ​​of the four 16×16 CUs obtained in process 5). This process is also performed for the remaining three 32×32 CUs in the same manner.

[0120] 7) Finally, the optimal partitioning structure of the CUs in the 64×64 block is determined by comparing the rate-distortion value of the 64×64 CU obtained in process 1) with the increase of the rate-distortion values ​​of the four 32×32 CUs obtained in process 6).

[0121] In the intra prediction mode, a prediction mode is selected as a PU unit, and prediction and reconstruction are performed on the selected prediction mode in an actual TU unit.

[0122] TU refers to a basic unit for performing actual prediction and reconstruction. TU includes a transform block (TB) for a luma component and TBs for two chroma components corresponding to the luma component.

[0123] exist Figure 3 In the example of , like in the example in which one CTU is split in a quadtree structure to generate CUs, TUs are hierarchically split in a quadtree structure from one CU to be coded.

[0124] Since the TU is partitioned in a quadtree structure, the TU partitioned from the CU may be partitioned into smaller and lower TUs. In HEVC, the size of the TU may be determined as one of 32×32, 16×16, 8×8, and 4×4.

[0125] Return to reference Figure 3 , the root node of the quadtree is assumed to be associated with a CU. The quadtree is partitioned until a leaf node is reached, and the leaf node corresponds to a TU.

[0126] This is described in more detail. A CU corresponds to a root node and has a minimum depth (ie, depth=0) value. Depending on the characteristics of an input image, a CU may not be split. In this case, a CU corresponds to a TU.

[0127] The CU can be split in a quadtree form. Therefore, a lower node with a depth of 1 (depth=1) is generated. In addition, a node (i.e., a leaf node) that belongs to the lower node with a depth of 1 and is no longer split corresponds to a TU. For example, in Figure 3 In (B), TU(a), TU(b), and TU(j) corresponding to nodes a, b, and j have been split once from the CU and have a depth of 1.

[0128] At least one of the nodes with depth 1 can be split again in the quadtree form. Therefore, a lower node with depth 2 (ie, depth=2) is generated. In addition, nodes (ie, leaf nodes) that belong to the lower node with depth 2 and are no longer split correspond to TUs. For example, in Figure 3 In (B), TU(c), TU(h), and TU(i) corresponding to nodes c, h, and i have been split twice from the CU and have a depth of 2.

[0129] In addition, at least one of the nodes with depth 2 may be split again in a quadtree form. Therefore, a lower node with depth 3 (ie, depth=3) is generated. In addition, a node (ie, leaf node) that belongs to the lower node with depth 3 and is no longer split corresponds to a CU. For example, in Figure 3 In (B), TU(d), TU(e), TU(f), and TU(g) corresponding to nodes d, e, f, and g have been split three times from the CU and have a depth of 3.

[0130] A TU having a tree structure may be hierarchically partitioned with predetermined maximum depth information (or maximum level information). In addition, each partitioned TU may have depth information. The depth information may include information about the size of the TU because it indicates the number of partitions and / or the extent of the TU.

[0131] Information indicating whether the corresponding TU has been split relative to one TU (e.g., split TU flag "split_transform_flag") may be transmitted to the decoder. The split information is included in all TUs except for the TU of the minimum size. For example, if the value of the flag indicating whether the TU has been split is "1", the corresponding TU is split into four TUs. If the value of the flag indicating whether the TU has been split is "0", the corresponding TU is no longer split.

[0132] Encoding / decoding of residual signal

[0133] An input image is differentiated by a block predicted by an intra prediction unit or an inter prediction unit, and information related to the prediction and a residual signal for differentiation are transmitted to reconstruct an image in a decoder. This will be described in more detail with reference to the drawings given below.

[0134] Figure 5 A method for encoding a residual signal is illustrated as an embodiment to which the present disclosure is applied.

[0135] refer to Figure 5 , the encoder (encoding device) transforms the residual signal (or residual block) (S501). The residual signal refers to a signal that is different from a signal predicted from an input image by an intra-frame prediction unit or an inter-frame prediction unit. The residual signal is a signal in the spatial domain and is transformed to the frequency domain to eliminate the correlation between signals and concentrate the energy in the low-frequency area. In this case, the encoder can generate transformation coefficients by applying transformation techniques such as DCT, DST, GBT, KLT, etc.

[0136] The encoder quantizes the transform coefficients (S502). The transformed signal is scaled and / or quantized to reduce the amplitude of the signal.

[0137] The encoder divides the transform block into sub-blocks (S503). The encoder may divide the transform block into sub-blocks having a predetermined constant size (eg, sub-blocks having a size of 4×4).

[0138] The encoder scans the quantized transform coefficients and encodes the coefficients according to the scanning order (S504). That is, the quantized transform coefficients are entropy-encoded according to a predetermined order and sent to the decoder.

[0139] Here, the scanning order may correspond to an upper right diagonal scanning order, a horizontal scanning order, a vertical scanning order, and the like.

[0140] The encoder may scan the transform coefficients in the reverse direction starting at the position of the last significant coefficient (the last non-zero coefficient) until the coordinate coefficient (0, 0) of the so-called DC coefficient is reached. In this case, the quantized transform coefficients in each sub-block may be scanned according to an upper right diagonal scanning order, and further, similarly, each sub-block may also be scanned according to an upper right diagonal scanning order.

[0141] In this case, information for identifying the position of the last significant coefficient in the transform block (ie, the position of a column and the position of a row) may be encoded.

[0142] The decoder can obtain the reconstructed image by performing the process inversely in the encoder. This will be described in more detail with reference to the drawings given below.

[0143] Figure 6 A method for decoding a residual signal as an embodiment to which the present disclosure is applied is illustrated.

[0144] refer to Figure 6 , the decoder (decoding device) decodes the coefficients (S601). That is, the decoder performs entropy decoding on the signal (ie, bit stream) output from the encoder to obtain quantized transform coefficients.

[0145] The decoder locates the quantized transform coefficients in a scanning order and inversely quantizes the transform coefficients (S602). That is, the decoder can arrange the quantized transform coefficients reconstructed by entropy decoding in a residual signal array according to a predetermined order.

[0146] As described above, the scanning order may correspond to an upper right diagonal scanning order, a horizontal scanning order, a vertical scanning order, and the like.

[0147] As described above, information indicating the position of the last significant coefficient (i.e., non-zero coefficient) according to the scanning order in the transform block can be signaled through the bitstream. The decoder parses the above information to derive the column position and row position of the last significant coefficient (i.e., non-zero coefficient) according to the scanning order in the transform block.

[0148] The decoder may scan the transform coefficients in the reverse direction starting at the position of the last significant coefficient (i.e., non-zero coefficient) until the coordinate (0,0) of the so-called DC coefficient is reached. In this case, the quantized transform coefficients in each sub-block may be scanned in the reverse order of the upper right diagonal scanning order, and further, each sub-block may also be similarly scanned in the reverse order of the upper right diagonal scanning order.

[0149] The decoder may inverse quantize the quantized transform coefficients according to the scanning order as described above.

[0150] The decoder inversely transforms the inversely quantized transform coefficients (S603). That is, the decoder may inversely transform the transform coefficients by applying an inverse transform technique and obtain a residual signal.

[0151] In this way, the coefficients reconstructed by entropy decoding can be arranged in the residual signal array according to the scanning order, and can be reconstructed into a residual signal in the spatial domain by inverse quantization and inverse transformation. In addition, the reconstructed residual signal is combined with the prediction signal and then output as a reconstructed image through a filtering process.

[0152] Example 1

[0153] In an embodiment of the present disclosure, the encoder / decoder may hierarchically configure the scanning order of the transform coefficients. In other words, the encoder / decoder may divide the transform block (or residual block or scan processing block) into coefficient groups, and then perform scanning in units of the divided coefficient groups. The coefficient group may be referred to as a sub-block, a sub-group, a sub-coefficient group, etc.

[0154] The encoder / decoder may scan the transform coefficients in each coefficient group while scanning the transform coefficients in units of coefficient groups partitioned from the transform block according to the scanning order. For example, the coefficient group may be a block having a size of 4×4. Figure 7 Describe this.

[0155] Figure 7 is a diagram illustrating a method for scanning transform coefficients in units of coefficient groups as an embodiment to which the present disclosure is applied.

[0156] refer to Figure 7 , assuming that the size of the current block (i.e., the transform block) is 8×8, and the coefficient groups are determined as 4×4 blocks. In this case, for example, the encoder / decoder may scan the coefficient groups in the order of CG1 701, CG3 703, CG2 702, and CG4 704. In addition, the encoder / decoder may scan the coefficients in each coefficient group according to a diagonal scanning order similar to the scanning order between coefficient groups.

[0157] When having Figure 7 The current block of size 8×8 is expressed as CG 0 (1) and a coefficient group having a size of 4×4 is expressed as CG 1 (1, k), k = 1, ..., 4 (where k represents the scanning order), Figure 7 The coefficient group of can satisfy the following equation 1.

[0158] [Equation 1]

[0159] CG 1 (1,1)=CG 1,CG 1 (1,2)=CG 3,CG 1 (1,3)=CG 2,CG 1 (1,4)=CG 4

[0160] Assume that the scan is from Figure 7 and the coefficient corresponding to the low frequency (i.e., the DC coefficient) in Equation 1, but encoding / decoding can be performed from the coefficient group scanned last in the actual encoding / decoding process. In addition, in this case, the coefficients in the coefficient group can be scanned by using the same scanning order, and encoding / decoding can be performed starting from the coefficient positioned last in the scanning order.

[0161] That is, when entropy coding is performed on transform coefficients, the coefficient groups may be encoded in the order of CG4 704, CG2 702, CG3 703, and CG1 701, and in this case, the DC component coefficient positioned at the upper left end may be encoded last.

[0162] In the present disclosure, an embodiment is described based on a method of first scanning a DC component at the upper left end as in Equation 1, but the present disclosure is not limited thereto, and a reverse scanning order may be applied. Even when a reverse scanning order is applied in an actual compilation process, the method proposed in the present disclosure may be applied. In this case, a reverse scanning order may be derived from a forward scanning order by using Equation 9 to be described below.

[0163] Figure 8 is a diagram showing a scanning order of transform coefficients for each coefficient group as an embodiment to which the present disclosure is applied.

[0164] refer to Figure 8 , the hierarchical relationship of the current block 801, coefficient groups 802, 803, 804 and 805 and the coefficients in each coefficient group can be represented as a tree structure. In this case, the encoder / decoder can perform scanning in order from the left to the right, and perform scanning according to a depth-first search method.

[0165] The leaf nodes having the tree structure represent the positions of coefficients in the current block 801. The position of each coefficient can be represented as a 1×2 row vector consisting of a row direction index and a column direction index.

[0166] In this disclosure, the coefficient group can be written as CG d (i0,i1,...,i d ). Here, i k(k=0, 1, ..., d) represents the scan index of the k-th depth of the corresponding coefficient group. In addition, d represents the depth of the corresponding coefficient group in the tree structure.

[0167] Fig. 9 and Fig.10 is a diagram for describing a method for determining a partition structure of a transform block and a scanning order of transform coefficients as an embodiment to which the present disclosure is applied.

[0168] refer to Fig. 9 , assuming that the current processing block is a block with a size of 16×16. In this case, the encoder / decoder may recursively (or hierarchically) split the current processing block (i.e., split the current processing block into layers with a depth of 2 or more) in order to perform scanning of the transform coefficients. Fig. 9 When the current processing block is recursively split as shown in (c), the split structure of the current processing block can be Fig.14 The tree representation shown in the figure.

[0169] refer to Fig.10 , the leaf nodes having the tree structure are scanned from the left to the right to perform scanning of all the transform coefficients. That is, the scanning order can be determined from the left to the right of the leaf nodes having the tree structure.

[0170] In the present disclosure, a leaf coefficient group is represented by Fig.10 The lowest depth coefficient group (i.e., Fig.10 The leaf coefficient group is no longer split into coefficient groups with a lower depth. In addition, in the block structure split into a tree structure, the depth increases as the number of splits increases. A node with a lower depth represents a node split from a node with a relatively large depth value, that is, a node with a higher depth.

[0171] CG d (i0,i1,...,i d ) can be expressed as shown in the following Equation 2.

[0172] [Equation 2]

[0173] where h = CG d (i0,i1,...,i d ) and w=CG d (i0,i1,...,i d )’s width.

[0174] refer to Figure 2 , each row vector [r i c i](i=1,...,h×w) represents the position of the transform coefficient in the h×w block. i represents the row index, and c i Represents the column index (1≤r i ≤h,1≤c i ≤w).

[0175] In this disclosure, it is assumed that the row index and column index of all 2D blocks start from 1. In Equation 2, τ(CG d (i0,i1,...,i d )) represents the scanning order of the coefficients in the corresponding coefficient group. That is, it can be scanned from [r1 c1] to [r hxw c hxw ] order to scan the coefficients at the corresponding positions.

[0176] As mentioned above, since τ(CG d (i0,i1,...,i d Each row vector of )) represents the position of the transform coefficient in a block of size h×w, so it will be used for the coefficient group CG with depth d d (i0,i1,...,i d ) of the transform block (i.e., CG 0 The row index offset and column index offset in (1) are added to form τ(CG d (i0,i1,...,i d )) to obtain the scanning order in the transform block. In this case, the row index offset and column index offset can be calculated by using the following equation 3.

[0177] [Equation 3]

[0178]

[0179] Referring to Equation 3, V rel (CG d+1 (i0,i1,...,i d+1 )) represents the CG of the current transform block d (i0,i1,...,i d ). For example, in the above Fig. 9 In, V rel (CG 2 (1,1,4)) = [8 8] can be established and V(CG 2 (1,2,2))=V(CG 1 (1,2))+V rel (CG 2 (1,2,2))=[12 0]+[0 8]=[12 8] can be established.

[0180] Define vector 1 as 1 = [1 1 … 1] T When CG is calculated, the following equation 4 can be used d (i0,i1,...,i d ) in the scanning order.

[0181] [Equation 4]

[0182] S(CG d (i0,i1,...,i d ))=τ(CG d (i0,i1,...,i d ))+1·V(CG d (i0,i1,...,i d ))

[0183] When CG d (i0,i1,...,i d ) is a leaf coefficient group, the scanning order in all blocks (i.e., transform blocks) can be calculated by using equation 4 above. That is, the scanning order in all blocks is equal to the scanning order in the coefficient group calculated by equation 2 above and the offset of the coefficient group calculated by equation 3 above (i.e., 1·V(CG d (i0,i1,...,i d ))) and.

[0184] When CG d (i0,i1,...,i d ) is not a leaf coefficient group, the following equation 5 can be used to calculate the scanning order in the transform block.

[0185] [Equation 5]

[0186] Where N = CG d (i0,i1,...,i d ) and d≥0

[0187] All Blocks CG 0 The scanning order of (1) can be expressed as S(CG 0 (i0)). In this case, S(CG 0 (i0)) can represent the scan index, and S(CG 0 Each row vector of (i0)) can represent the position of the corresponding scan index in all blocks.

[0188] Fig.11 is a diagram illustrating one example of the positions of coefficients in a representation block as an embodiment to which the present disclosure is applied.

[0189] refer to Fig.11 , the position of coefficients in a block can be represented by using raster scan order (i.e., row-major order and lexicographic order). Fig.11 As shown in FIG, index values ​​can be assigned to the positions of the respective coefficients according to the scanning order. Fig.11 The index value of the position of the coefficient can be expressed as shown in the following Equation 6.

[0190] [Equation 6]

[0191] Where w = CG 0 The width of (i0) and d ≥ 0

[0192] When an index representing a coefficient position is assigned in the position of a transform coefficient in a block according to a random scanning order including the above-mentioned raster scanning order, the scanning order may be expressed as shown in Equation 7 below.

[0193] [Equation 7]

[0194] Where f:R L×2 →R L ,L=(CG 0 (height of (i0))×(CG 0 (width of i0)

[0195] when When , Equation 7 can be summarized as shown in Equation 6.

[0196] The scanning order proposed in the present disclosure may be described by the scheme described in the above Equations 2 to 7. When the scanning order is thus described by the above scheme, the scanning order may be determined by the following components.

[0197] 1) Recursively split the coefficient groups at each depth (i.e., 0 or greater)

[0198] 2) Determine the scanning order of the partition coefficient group

[0199] 3) Determine the scanning order of the transform coefficients of the leaf coefficient group

[0200] When the components are determined, the scanning order may be determined by using Equations 2 to 7. Therefore, the present disclosure proposes a method of configuring an appropriate scanning order by determining the components.

[0201] A set including all applicable scan order matrices can be expressed as shown in Equation 8 below.

[0202] [Equation 8]

[0203]

[0204] here, Denotes a set where all scan order matrices applicable to these blocks are collected. In addition, blocks can have various sizes or shapes (e.g., 4×4, 4×8, 16×4, 16×16, 32×8, etc.). In addition, S(CG) derived from Equation 5 above can be 0 (i0)) can be used as The elements are included.

[0205] In an embodiment, the encoder / decoder may select a block by using information about neighboring blocks of the currently processed block. The information about the neighboring blocks may be, for example, the number of non-zero coefficients existing in the left block or the upper block or the position of the last non-zero coefficient. In addition, the scanning order of the current processing block may be determined by using information about the block encoded immediately before the current processing block.

[0206] In an embodiment, the encoder / decoder may determine (or apply) a scanning order based on the size or shape of the current processing block. is applied to blocks of size 4×8 and can be Applied to blocks with size 8×16.

[0207] In an embodiment, the encoder / decoder may select a set of scanning orders suitable for blocks of a certain size or shape (ie, In this case, the encoder may signal to the decoder an index of the scanning order selected in units of sequence, picture, slice, CTU, coding block, or transform block (or transform unit).

[0208] When entropy encoding is performed on transform coefficients, a reverse scanning order may be applied in addition to the forward scanning order. The reverse scanning order may be calculated by using Equation 9 below.

[0209] [Equation 9]

[0210] When CG d (i0,i1,...,i d )=leafCG, S B (CG d (i0,i1,...,i d ))=τ B (CG d (i0,i1,...,i d ))+1·V(CG d(i0,i1,...,i d )),otherwise

[0211]

[0212] , where N = CG d (i0,i1,...,i d ) and d≥0

[0213] Here, P represents a permutation matrix having only anti-diagonal components with a value of 1. A scanning order opposite to the forward scanning order can be derived by using the permutation matrix.

[0214] Example 2

[0215] In an embodiment of the present disclosure, the encoder / decoder may determine the scanning order of each coefficient in the coefficient group based on the size or shape of the coefficient group. That is, the encoder / decoder may determine the scanning order of the coefficients in the coefficient group of equation 2 above according to the size or shape of the block.

[0216] Fig.12 and Fig.13 is a diagram illustrating a scanning order depending on positions of transform coefficients in a block according to an embodiment of the present disclosure.

[0217] refer to Fig.12 , the encoder / decoder can scan the coefficients in the order of increasing d values ​​in a diagonal scanning order. Here, d represents the distance of the corresponding coefficient based on the upper left coefficient. The encoder / decoder can scan the coefficients at the position with the same d value from the lower left side to the upper right side.

[0218] Here, the d value can be determined as d=dx+dy, and as Fig.12 As shown in (b), dx and dy represent the horizontal increment (or horizontal coordinate) and the vertical increment (or vertical coordinate), respectively.

[0219] As described above, the encoder / decoder may apply the reverse scanning order to perform entropy coding on the transform coefficients. In this case, the coefficients where the d value is 6 to the coefficients where the d value is 0 may be scanned and in the same order as Fig.12 The arrows illustrated in (a) scan in opposite directions from the upper right side to the lower left side. In the present disclosure, the forward scanning order is mainly described, but when performing entropy encoding, the encoder / decoder can follow the reverse scanning order or the forward scanning order.

[0220] Positions with the same d value may correspond to positions of coefficients with equal phase in frequency. Thus, the encoder / decoder may assign a random order to the coefficients at positions with the same d value. For example, the encoder / decoder may scan row positions in the order of [4 1], [3 2], [2 3], and [1 4] (each position is expressed by a row vector), [1 4], [2 3], [3 2], and [4 1], or [2 3], [3 2], [1 4], and [4 1], in which row positions, Fig.12 The d value of (a) is 3.

[0221] In addition, the encoder / decoder can be configured as Fig.13 The figure shows each anti-diagonal line scanned from right to left, unlike Fig.12 Example of (a).

[0222] Table 1 below shows an example of a pseudo code for determining a diagonal scan order.

[0223] [Table 1]

[0224]

[0225] Referring to Table 1, the encoder / decoder may determine the scanning order by assigning scanning indexes from the coefficient at the most lower left position to the coefficient at the most upper right position while increasing the d value from 0 to 1. Here, sypos and sxpos represent parameters indicating values ​​obtained by subtracting 1 from the row index and the column index, respectively. That is, it is assumed that sypos and sxpos start from 0.

[0226] In Table 1, it is assumed that the diagonal line specified by the d value is scanned from the lower left side, but the code can be configured to scan the diagonal line from the upper right side, such as Fig.13 Described in .

[0227] In the above Table 1, the "while loop" determines the parameter (or matrix) representing the scan information according to the scan order relative to the diagonal specified by the d value. The parameter can be a parameter (scan_order) representing the scan order or a parameter (scan_pos) representing the scan position. The scan index assigned to each position of the block (i.e., the component of the scan_order matrix) is stored in the scan_order matrix. In addition, in the scan_pos matrix, each row index corresponds to the scan index, and the coefficient position in the block is stored in each row in the form of a row vector.

[0228] When a scanning order is assigned to the positions of coefficients by an existing method (eg, HEVC), a horizontal or vertical frequency increase rate may not be effectively reflected when a current processing block is a non-square block.

[0229] Specifically, when a separable transform is applied to a non-square block, the length of the transform basis vector in the vertical direction and the length of the transform basis vector in the horizontal direction may be different from each other. For example, when the current processing block is a block having a size of 2N×N, a DCT having a size of 2N×2N may be applied to the vertical direction, and a DCT having a size of N×N may be applied to the horizontal direction. In this case, whenever the vertical coordinate of the coefficient increases by 1, the frequency increases by 1 / (2N)*π in the height direction, and whenever the horizontal coordinate of the coefficient increases by 1, the frequency increases by 1 / (N)*π in the width direction.

[0230] When describing the method proposed in the present disclosure, it is mainly described that the M×N block is a block with a height of M and a width of N, but the present disclosure is not limited thereto, and the M×N block may represent a block with a width and a height of M and N, respectively.

[0231] In the case of a non-square block, when the scanning order is similarly applied to the position of the coefficient by the existing method, the frequency increase rate may not be effectively reflected in the horizontal direction or the vertical direction. As a result, the scanning from the high-frequency component to the low-frequency component may not be performed normally, and the compression performance of the image may be reduced.

[0232] Therefore, the present disclosure proposes a method for assigning a scanning order to coefficients at various positions in a non-square block to solve the problem and effectively reflect a frequency increase rate in a horizontal direction or a vertical direction.

[0233] Even for non-square blocks, this can be done by using the above Fig.12 The scanning order is obtained by the method described in Table 1. For example, when the current processing block is a block with a size of 4×8, the height and width can be configured as 4 and 8, respectively, as input parameters of the function, and when the function of Table 1 is executed, the corresponding scanning order can be obtained.

[0234] Fig.14 and Fig.15 is a diagram for describing a method for determining a scanning order of coefficients of a non-square block as an embodiment to which the present disclosure is applied.

[0235] As described above, when a separable transform is applied to a non-square block, the length of a transform basis vector in a vertical direction (ie, a row direction) and the length of a transform basis vector in a horizontal direction (ie, a column direction) may be different from each other.

[0236] refer to Fig.14, assuming that the current processing block is a block having a 4×8 size. In this case, a 4×4-sized DCT may be applied to the vertical direction, and an 8×8-sized DCT may be applied to the horizontal direction. When it is assumed that the transform coefficient represents a low-frequency component when the transform coefficient is positioned at the upper left end and the transform coefficient represents a high-frequency component when the transform coefficient is positioned at the lower right end, Fig.14 In the case of the 4×8 block of (a), the frequency increment in the vertical direction (or height direction) may be twice the frequency increment in the horizontal direction (or width direction).

[0237] Therefore, dx and dy depending on the position of the transform coefficient can be as follows Fig.14 (b) is configured as shown. In addition, the encoder / decoder can configure the scanning order in the order in which the d value increases in units of coefficients at positions having the same d value (in this case, d=dx+dy). In the case of a reverse scanning order, the encoder / decoder can configure the scanning order in the order in which the d value decreases.

[0238] In the present disclosure, a scan line may refer to coefficients at positions having the same d value. That is, coefficients having the same d value may be included in the same scan line.

[0239] A random scan order can be assigned between coefficients belonging to the same scan line. When scanning coefficients from left to right, scan indices can be assigned such as Fig.14 The coefficients at various positions are shown in (c).

[0240] refer to Fig.15 , assuming that the current processing block is a block having a size of 8×4. The encoder / decoder may determine the scanning order by configuring the d value in the same method as the above method, except that the horizontal frequency increment and the vertical frequency increment are different from each other.

[0241] Table 2 below shows the Fig.14 and Fig.15 An example of pseudo code for the method described in to obtain the scan order.

[0242] [Table 2]

[0243]

[0244]

[0245] Referring to Table 2, sh is a parameter representing a vertical increment (ie, dy), and sw is a parameter representing a horizontal increment (ie, dx). The other parameters are the same as those in Table 1 above. In addition, the floor function returns an integer value whose maximum size is equal to or less than the number given as a factor, and the round function returns an integer value obtained by rounding the number given as a factor. The encoder / decoder configures the sh value and the sw value by using the ratio of the width and height of the current processing block, and assigns the scan index to the coefficient at the lower leftmost position to the coefficient at the upper rightmost position while increasing the d value from 0 to 1 to determine the scan order.

[0246] In an embodiment of the present disclosure, the encoder / decoder may group coefficients in units of a specific number of coefficients, and then scan the coefficients by applying various scanning orders. Fig.16 Describe this.

[0247] Fig.16 and Fig.17 is a diagram for describing a method for determining a scanning order of coefficients of a non-square block in units of superpixels as an embodiment to which the present disclosure is applied.

[0248] refer to Fig.16 , assuming that the current processing block is a block having a size of 4×8. The encoder / decoder may divide the current processing block in units of superpixels. Here, a superpixel may represent a set of a specific number of coefficients (or pixels), and the specific number may be determined according to the ratio of the width to the height of the current block. A superpixel may be referred to as a subblock, a subgroup, a coefficient group, a subcoefficient group, etc. Because the ratio of the width to the height of the current block is 2, a superpixel may be formed by two pixels.

[0249] The encoder / decoder can use superpixels to configure square blocks by using the above Figures 7 to 13 The method described in determines the scanning order in superpixels and scans the coefficients in a random order within the superpixel.

[0250] The encoder / decoder may apply a scanning order on a square block having a size of 4×4 in units of superpixels in a diagonal scanning order, such as Fig.16 As illustrated in , the scanning order is applied from left to right in each superpixel.

[0251] refer to Fig.17 , assuming that the current processing block is a block with a size of 8×4. Similarly, the encoder / decoder can divide the current processing block in units of superpixels. Because the height is twice the width, the encoder / decoder can configure superpixels by grouping two coefficients in the vertical direction (or height direction).

[0252] Table 3 below shows the Fig.16 and 17 An example of pseudo code for the method described in to obtain the scan order.

[0253] [Table 3]

[0254]

[0255]

[0256] Referring to Table 3, splen is a parameter indicating the length of a superpixel. splen can be determined based on the width and height of the current processing block. For example, when the height is greater than the width, the superpixel can be determined as a superpixel that is longer in the vertical direction, and when the width is greater than the height, the superpixel can be determined as a superpixel that is longer in the horizontal direction. In addition, norm_length is a parameter indicating the length of one side of a block (i.e., a square block) configured in units of superpixels. In Table 3 above, the "for-loop" in the "while loop" assigns scan indices to coefficients at various positions in the superpixel.

[0257] The encoder / decoder may apply various scanning orders in addition to this scanning order. For example, the encoder / decoder may obtain information on the number of non-zero coefficients of coefficients at respective positions by using statistics of block transform coefficients having a specific size (e.g., 4×8, 16×8, 32×32, etc.). In addition, the encoder / decoder may determine the scanning order of coefficients in descending order (or ascending order) of the corresponding number based on the number of non-zero coefficients obtained.

[0258] When CG d (i0,i1,...,i d ) has a height and a width of h and w respectively, as shown in Equation 2 above, the encoder / decoder can represent a set of scanning orders applicable to an h×w block as shown in Equation 10 below.

[0259] [Equation 10]

[0260]

[0261] here, represents the set of applicable scan orders. In addition, can represent the scanning order and can correspond to the above Figures 7 to 17 The scanning order described in Equation 2 is τ(CG d (i0,i1,...,i d ))satisfy

[0262] Example 3

[0263] In an embodiment of the present disclosure, a method for determining a scanning order of coefficient groups is proposed. For example, an encoder / decoder may determine Fig. 9 (c) constitutes CG 1 The scanning order of the four coefficient groups of (1,1).

[0264] The encoder / decoder can scan the scan groups in a diagonal scan order, such as Fig. 9 In addition, the encoder / decoder may apply the scanning methods described in Examples 1 and 2 above when determining the scanning order between coefficient groups, and apply the scanning order having a random order.

[0265] Fig.18 is a diagram illustrating a method for determining a scanning order of coefficient groups as an embodiment to which the present disclosure is applied.

[0266] refer to Fig.18 , the encoder / decoder can be configured to similarly use the scanning order of coefficient groups and the scanning order of coefficients in coefficient groups, or to use each scanning order separately.

[0267] As an example, see Fig.18 (a) The encoder / decoder may partition the coefficient groups into blocks having a size of 4×2, apply the scanning order generated by using the above Table 2 between the coefficient groups, and apply the scanning order generated by using the above Table 1 in each coefficient group.

[0268] As another example, refer to Fig.18 (b), the encoder / decoder may partition the coefficient groups into blocks having a size of 2×4 and apply a scanning order generated by using the above-described Table 2 similar to a scanning order between coefficient groups and a scanning order between coefficients in a coefficient group.

[0269] As another example, refer to Fig.18 (c) The encoder / decoder may partition the coefficient groups into blocks having a size of 2×4, apply the scanning order generated by using the above Table 2 between the coefficient groups, and apply the scanning order generated by using the above Table 3 in each coefficient group.

[0270] Fig.19 A method for encoding an image according to an embodiment of the present disclosure is illustrated.

[0271] The encoder performs transformation and quantization on the residual signal of the current processing block to generate a quantized transformation block (S1901). Figure 5 Step S1901 is performed by performing steps S501 and S502 described in the specification.

[0272] The encoder divides the quantized transform block into a plurality of coefficient groups (S1902).

[0273] As in the above Fig. 9 and Fig.10 As described in , the encoder may recursively (or hierarchically) partition the current processing block in order to scan the transform coefficients. For example, partitioning the quantized transform block into a plurality of coefficient groups may include partitioning the quantized transform block hierarchically and partitioning the quantized transform block into coefficient groups with lower depths, and the second scanning order may include a scanning order among the coefficient groups with respective depths partitioned from the quantized transform block.

[0274] In addition, as above Fig.16 and 17 As described in , the encoder may group coefficients in units of a specific number of coefficients, and then scan the coefficients by applying various scanning orders. That is, when the quantized transform block is a non-square block, the quantized transform block may be divided into coefficient groups consisting of a specific number of coefficients, and the specific number may be determined according to the ratio of the width and height of the quantized transform block.

[0275] The encoder determines a first scanning order representing a scanning order between coefficients of a coefficient group (S1903).

[0276] The encoder can be Figures 12 to 18 For example, step S1903 may include: determining the distance values ​​of the coefficients of the coefficient group based on the upper left coefficient of the coefficient group; and assigning the scanning indexes to the coefficients with the same distance value according to a predetermined order while sequentially increasing the distance value.

[0277] Further, as above Fig.14 and 15 As described in , the encoder may assign a scanning order to coefficients at various positions in a non-square block so as to effectively reflect a frequency increase rate in the horizontal direction or the vertical direction. For example, the step of determining the distance value of each coefficient of the coefficient group may include: determining a horizontal increment representing an increase amount of a coordinate value in the horizontal direction of each coefficient and a vertical increment representing an increase amount of a coordinate value in the vertical direction based on a ratio of a width and a height of the coefficient group, and the distance value of each coefficient may be determined by using a value obtained by summing the horizontal increment and the vertical increment.

[0278] Furthermore, when the width of the coefficient group is greater than the height, the horizontal increment may be determined to be 1 and the vertical increment may be determined to be 2, and when the height of the coefficient group is greater than the width, the horizontal increment may be determined to be 2 and the vertical increment may be determined to be 2.

[0279] The first scanning order may be predefined (or configured) by the encoder, and the encoder may send the first scanning order to the decoder in units of a picture, a slice, a CTU, a coding unit, or a transform unit.

[0280] In addition, the encoder may determine a second scanning order representing a scanning order among the plurality of coefficient groups. In this case, step S1903 may include determining the second scanning order, and the encoder may apply the above Figures 12 to 18 The second scanning order may be determined by the method described in . The second scanning order may be predefined (or configured) by the encoder, and the encoder may send the second scanning order to the decoder in units of pictures, slices, CTUs, coding units, or transform units.

[0281] The encoder entropy encodes coefficients of the quantized transform block according to a first scanning order and a second scanning order indicating a scanning order among a plurality of coefficient groups (S1904).

[0282] Fig. 20 A method for decoding an image according to an embodiment of the present disclosure is illustrated.

[0283] The decoder divides the current processing block into a plurality of coefficient groups (S2001).

[0284] As above Fig. 9 and Fig.10 As described above, the decoder may recursively (or hierarchically) partition the current processing block in order to scan the transformation coefficients. For example, partitioning the current processing block into a plurality of coefficient groups may include: partitioning the current processing block hierarchically, and partitioning the current processing block into coefficient groups with a lower depth, and the second scanning order may include a scanning order among the coefficient groups with each depth partitioned from the current processing block.

[0285] In addition, as above Fig.16 and Fig.17 As described in , the decoder can group coefficients in units of a specific number of coefficients, and then scan the coefficients by applying various scanning orders. That is, when the current processing block is a non-square block, the current processing block can be divided into coefficient groups consisting of a specific number of coefficients, and the specific number can be determined according to the ratio of the width and height of the current processing block.

[0286] The decoder determines a first scanning order indicating a scanning order between coefficients of a coefficient group (S2002).

[0287] The decoder can be Figures 12 to 18For example, step S2002 may include determining the distance values ​​of the coefficients of the coefficient group based on the upper left coefficient of the coefficient group; and assigning the scanning indexes to the coefficients with the same distance value according to a predetermined order, while sequentially increasing the distance value.

[0288] In addition, as above Fig.14 and 15 As described in , the decoder may assign a scanning order to coefficients at various positions in a non-square block so as to effectively reflect a frequency increase rate in the horizontal direction or the vertical direction. For example, determining the distance value of each coefficient of a coefficient group may include: determining a horizontal increment representing an increase amount of a coordinate value in the horizontal direction of each coefficient and a vertical increment representing an increase amount of a coordinate value in the vertical direction based on a ratio of a width and a height of the coefficient group, and the distance value of each coefficient may be determined by using a value obtained by summing the horizontal increment and the vertical increment.

[0289] Furthermore, when the width of the coefficient group is greater than the height, the horizontal increment may be determined to be 1 and the vertical increment may be determined to be 2, and when the height of the coefficient group is greater than the width, the horizontal increment may be determined to be 2 and the vertical increment may be determined to be 2.

[0290] The first scanning order may be pre-defined (or configured) by the decoder, and the decoder may send the first scanning order to the decoder in units of pictures, slices, CTUs, coding units, or transform units.

[0291] In addition, the decoder may determine a second scanning order representing a scanning order among the plurality of coefficient groups. In this case, step S2002 may include determining a second scanning order, and the decoder may determine the second scanning order by applying the above Figures 12 to 18 The second scanning order may be determined by the method described in . The second scanning order may be predefined (or configured) by the decoder and may be sent from the encoder to the decoder in units of pictures, slices, CTUs, coding units, or transform units.

[0292] The decoder performs entropy decoding on the bit stream output from the encoder to generate quantized transform coefficients (S2003). Figure 6 Step S2003 is performed according to step S601 described in the specification.

[0293] The decoder arranges coefficients of the quantized transform block according to a first scanning order and a second scanning order indicating a scanning order among a plurality of coefficient groups to generate a quantized transform block of a current processing block (S2004).

[0294] In addition, the decoder can obtain the transform block by dequantizing the quantized transform block. The dequantization method can adopt a known technology, and its detailed description will be omitted. In the image decoding method according to the above embodiment, some steps can be omitted or added, and the image decoding method is not limited even in the described order. In addition, steps S2003 and S2004 can be performed separately or simultaneously.

[0295] Fig.21 A device for decoding an image according to an embodiment of the present disclosure is illustrated.

[0296] exist Fig.21 , for ease of description, the coefficient group partition unit 2101, the scanning order determination unit 2102, the transform coefficient generation unit 2103 and the transform block generation unit 2104 are illustrated as separate blocks, but the coefficient group partition unit 2101, the scanning order determination unit 2102, the transform coefficient generation unit 2103 and the transform block generation unit 2104 may be implemented as components included in an encoder and / or a decoder.

[0297] refer to Fig.21 According to the device for decoding an image of this embodiment, the above Figures 5 to 20 The functions, processes and / or methods proposed in the above are also implemented, and the scaling and transformation processes described below are also performed. Specifically, the decoding device can be configured to include a coefficient group partitioning unit 2101, a scanning order determination unit 2102, a transformation coefficient generation unit 2103 and a transformation block generation unit 2104.

[0298] exist Fig.21 The detailed components of the apparatus for decoding an image illustrated in are merely one example, and some of the illustrated detailed components may be included in another detailed component and implemented together, and another component not illustrated may be added and implemented together.

[0299] The coefficient group division unit 2101 divides the current processing block into a plurality of coefficient groups.

[0300] As in the above Fig. 9 and Fig.10 As described in , the coefficient group partitioning unit 2101 may recursively (or hierarchically) partition the current processing block in order to scan the transformation coefficients. For example, the coefficient group partitioning unit 2101 hierarchically partitions the current processing block to partition the current processing block into coefficient groups with lower depths, and the second scanning order may include a scanning order among coefficient groups with respective depths partitioned from the current processing block.

[0301] Further, as above Fig.16 and Fig.17As described in , the coefficient group division unit 2101 can group coefficients in units of a specific number of coefficients, and then scan the coefficients by applying various scanning orders. That is, when the current processing block is a non-square block, the current processing block can be divided into coefficient groups consisting of a specific number of coefficients, and the specific number can be determined according to the ratio of the width and height of the current processing block.

[0302] The scanning order determination unit 2102 determines a first scanning order representing a scanning order between coefficients of a coefficient group.

[0303] The scanning order determination unit 2102 can be determined by applying the above Figures 12 to 18 For example, the scanning order determining unit 2102 may determine the distance values ​​of the coefficients of the coefficient group based on the upper left coefficient of the coefficient group, and assign the scanning indexes to the coefficients with the same distance value according to a predetermined order and increase the distance value in sequence.

[0304] In addition, as mentioned in 14 above Fig.15 As described in , the scanning order determination unit 2102 may assign scanning orders to coefficients at various positions in the non-square block so as to effectively reflect the frequency increase rate in the horizontal direction or the vertical direction. For example, the scanning order determination unit 2102 may determine a horizontal increment representing an increase amount of the coordinate value in the horizontal direction of each coefficient and a vertical increment representing an increase amount of the coordinate value in the vertical direction based on a ratio of a width to a height of a coefficient group, and may determine the distance value of each coefficient by using a value obtained by adding the horizontal increment and the vertical increment.

[0305] Furthermore, when the width of the coefficient group is greater than the height, the horizontal increment may be determined to be 1 and the vertical increment may be determined to be 2, and when the height of the coefficient group is greater than the width, the horizontal increment may be determined to be 2 and the vertical increment may be determined to be 2.

[0306] The first scanning order may be pre-defined (or configured) by the decoder, and the decoder may send the first scanning order to the decoder in units of pictures, slices, CTUs, coding units, or transform units.

[0307] In addition, the scanning order determining unit 2102 may determine a second scanning order representing a scanning order among the plurality of coefficient groups. In other words, the scanning order determining unit 2102 may include determining a second scanning order, and the scanning order determining unit 2102 may determine a second scanning order by applying the above Figures 12 to 18 The second scanning order is determined by the method described in . In addition, the second scanning order may be predefined (or configured) by the decoder and may be sent from the encoder to the decoder in units of pictures, slices, CTUs, coding units, or transform units.

[0308] The transform coefficient generation unit 2103 performs entropy decoding on the bit stream output from the encoder to generate quantized transform coefficients. Figure 6 The same method as step S601 described in.

[0309] The transform block generation unit 2104 arranges coefficients of the quantized transform block according to the first scanning order and a second scanning order indicating a scanning order among a plurality of coefficient groups to generate a quantized transform block of the current processing block.

[0310] In the above-mentioned embodiments, the components and features of the present disclosure are combined in a predetermined form. Unless otherwise clearly stated, each component or feature should be regarded as an option. Each component or feature can be implemented as not associated with other components or features. In addition, the embodiments of the present disclosure can be configured by associating some components and / or features. The order of the operations described in the embodiments of the present disclosure can be changed. Some components or features of any embodiment can be included in another embodiment, or replaced by components and features corresponding to another embodiment. It is obvious that claims that are not clearly cited in the claims are combined to form embodiments, or are included in new claims by modification after application.

[0311] The embodiments of the present disclosure may be implemented by hardware, firmware, software or a combination thereof. In the case of being implemented by hardware, according to the hardware implementation, the exemplary embodiments described herein may be implemented by using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0312] In the case of implementation by firmware or software, the embodiments of the present disclosure may be implemented in the form of modules, procedures, functions, etc. that perform the above functions or operations. The software code may be stored in a memory and executed by a processor. The memory may be located inside or outside the processor and may send data to or receive data from the processor by various means.

[0313] It is obvious to those skilled in the art that the present disclosure may be embodied in other specific forms without departing from the essential features of the present disclosure. Therefore, the foregoing detailed description should not be interpreted as limiting in all aspects, but should be considered as illustrative. The scope of the present disclosure should be determined by the reasonable interpretation of the attached claims, and all modifications within the equivalent scope of the present disclosure are included in the scope of the present disclosure.

[0314] [Industrial Applicability]

[0315] In the above, the preferred embodiments of the present disclosure are disclosed for illustrative purposes, and hereinafter, those skilled in the art will modify, change, replace or add various other embodiments within the technical spirit and technical scope of the present disclosure disclosed in the appended claims.

Claims

1. A method for decoding an image, the method comprising: Splitting a current block into sub-blocks, wherein the current block is a non-square block and the sub-blocks are non-square sub-blocks; as well as generating a transform block for the current block based on a first scanning order and a second scanning order, wherein the first scanning order represents a scanning order among coefficients of the subblocks, and wherein the second scanning order represents a scanning order between the subblocks; Wherein, based on the width of the current block being smaller than the height of the current block, the width of the sub-block is equal to the width of the current block, and the height of the sub-block is larger than the width of the sub-block, and Wherein, based on the fact that the height of the current block is smaller than the width of the current block, the height of the sub-block is equal to the height of the current block, and the width of the sub-block is larger than the height of the sub-block.

2. The method according to claim 1, wherein: The generating the transform block comprises: determining the first scanning order; and Wherein, determining the first scanning order includes: determining a distance value of a coefficient within the sub-block based on an upper left coefficient of the sub-block, and Scan indices are allocated to coefficients having the same distance value according to a predetermined order while sequentially increasing the distance value.

3. The method according to claim 2, wherein: Determining the distance value of the coefficients within the sub-block includes: determining a horizontal increment representing an increase in a coordinate value of each coefficient in a horizontal direction and a vertical increment representing an increase in a coordinate value in a vertical direction based on a ratio between a width and a height of the sub-block, wherein the distance value of the coefficient is determined by a value obtained by summing the horizontal increment and the vertical increment.

4. The method according to claim 3, wherein: Based on the width of the sub-block being greater than the height, the horizontal increment is determined to be 1 and the vertical increment is determined to be 2, and Wherein, based on the fact that the height of the sub-block is greater than the width, the horizontal increment is determined to be 2, and the vertical increment is determined to be 1.

5. The method according to claim 1, wherein: Splitting the current block into sub-blocks comprises: hierarchically partitioning the current block into sub-blocks with lower depths, and The second scanning order includes a scanning order between sub-blocks divided from the current block and having corresponding depths.

6. The method according to claim 1, wherein: The current block is divided into a plurality of coefficient groups having a specific number of coefficients, and The specific number is determined based on a ratio of a width to a height of the current block.

7. A method for encoding an image, the method comprising: Splitting a current block into sub-blocks, wherein the current block is a non-square block, wherein the sub-blocks are non-square sub-blocks; and quantizing and entropy encoding a transform block of the current block based on a first scanning order and a second scanning order, wherein the first scanning order represents a scanning order in coefficients of the sub-block, The second scanning order represents the scanning order between the sub-blocks. Wherein, based on the width of the current block being smaller than the height of the current block, the width of the sub-block is equal to the width of the current block, and the height of the sub-block is larger than the width of the sub-block, and Wherein, based on the fact that the height of the current block is smaller than the width of the current block, the height of the sub-block is equal to the height of the current block, and the width of the sub-block is larger than the height of the sub-block.

8. A method for transmitting data of a bit stream including an image, the method comprising: Generating the bitstream of the image, wherein the bitstream is generated by performing the following steps: dividing a current block into sub-blocks, wherein the current block is a non-square block, wherein the sub-blocks are non-square sub-blocks; and performing quantization and entropy coding on a transform block of the current block based on a first scanning order and a second scanning order, wherein the first scanning order represents a scanning order among coefficients of the sub-blocks, wherein the second scanning order represents a scanning order among the sub-blocks, wherein based on the width of the current block being smaller than the height of the current block, the width of the sub-block is equal to the width of the current block, and the height of the sub-block is greater than the width of the sub-block, and wherein based on the height of the current block being smaller than the width of the current block, the height of the sub-block is equal to the height of the current block, and the width of the sub-block is greater than the height of the sub-block; and The data including the bitstream is transmitted.

Images