VIDEO ENCODING AND DECODING EQUIPMENT AND VIDEO DECODING EQUIPMENT DEVICES

Abstract

The invention proposes a video decoding device for decoding a target block using internal prediction. The device includes decoding, from the bitstream, boundary information that defines at least one boundary for partitioning the target block. The boundary information allows partitioning the target block into multiple non-rectangular blocks. The device also includes defining internal prediction modes for non-rectangular blocks based on boundary information; generating a predicted block of the target block by performing internal prediction on each non-rectangular block using the internal prediction modes; reconstructing a redundant block of the target block from the bitstream; and reconstructing the target block by adding the predicted block and the redundant block. The invention also relates to an encoding device and a device providing video decoding capabilities.

Description

Video encoding and decoding using arbitrary block partitioning

[0001] The present invention relates to video encoding and decoding using arbitrary block division.

[0002] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0003] Therefore, when storing or transmitting video data, an encoder is used to compress the video data and store or transmit it, and a decoder receives the compressed video data, decompresses it, and plays it back. Examples of such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0004] However, as picture sizes, resolutions, and frame rates continue to increase, and the amount of data to be encoded also grows, new compression technologies with better encoding efficiency and improved image quality are required. In particular, compression technologies capable of more efficiently encoding pictures with complex textures, such as pictures containing edges (boundaries between objects) in various directions due to the presence of various objects, are required.

[0005] The present disclosure proposes a method for efficiently encoding or decoding a video containing edges in various directions. More specifically, it proposes a prediction and transformation method for a block containing edges that are neither horizontal nor vertical.

[0006] One aspect of the present invention provides an image decoding method for decoding a target block using intra prediction. The decoding method comprises the steps of: decoding boundary information specifying at least one boundary for dividing the target block from a bitstream, the boundary information allowing dividing the target block into a plurality of non-rectangular blocks; determining intra prediction modes for the non-rectangular blocks based on the boundary information; generating a prediction block of the target block by intra-predicting each of the non-rectangular blocks using the intra prediction modes; reconstructing a residual block of the target block from the bitstream; and reconstructing the target block by adding the prediction block and the residual block.

[0007] The step of restoring the residual block may include the steps of: restoring a plurality of rectangular transform coefficient blocks from the bitstream; generating residual sub-blocks by inversely transforming each of the transform coefficient blocks; and restoring the residual block by dividing the target block into a plurality of regions based on a boundary specified by the boundary information, the number of the transform coefficient blocks, and the size of each of the transform coefficient blocks, and rearranging residual signals within each residual sub-block into a corresponding region. Here, the plurality of regions include a boundary region including the boundary line and formed near the boundary line, and one or more non-boundary regions not including the boundary line.

[0008] Another aspect of the present invention provides an image encoding method for encoding a target block using intra prediction. The encoding method comprises the steps of: dividing the target block using at least one boundary line, the boundary line allowing the target block to be divided into a plurality of non-rectangular blocks; determining intra prediction modes for the non-rectangular blocks based on the boundary line; generating a prediction block of the target block by intra-predicting each of the non-rectangular blocks; generating a residual block of the target block by subtracting the prediction block from the target block; and encoding boundary line information for specifying the boundary line and residual signals within the residual block.

[0009] The step of encoding residual signals within the residual block includes the step of dividing the target block into a plurality of regions, the plurality of regions including a boundary region formed near the boundary region and one or more non-boundary regions not including the boundary region; and, for each of the plurality of regions, the step of rearranging residual signals within the region to generate a rectangular residual sub-block, and transforming the residual sub-block.

[0010] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0011] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0012] Figure 3 is a diagram showing multiple intra prediction modes.

[0013] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0014] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0015] FIG. 6 is an example diagram for geometrically dividing a target block according to one aspect of the present disclosure.

[0016] FIG. 7 is an exemplary diagram showing various predefined boundaries according to one aspect of the present disclosure.

[0017] FIG. 8 is an exemplary diagram illustrating a method of configuring reference pixels for each subblock according to one aspect of the present disclosure.

[0018] FIG. 9 is another exemplary diagram illustrating a method of configuring reference pixels for each sub-block according to one aspect of the present disclosure.

[0019] FIG. 10 is an exemplary diagram illustrating a method for determining an intra prediction mode of each subblock according to one aspect of the present disclosure.

[0020] FIG. 11 is an exemplary diagram illustrating a method for encoding an intra prediction mode of each subblock according to one aspect of the present disclosure.

[0021] FIG. 12 is an exemplary diagram illustrating a method for transforming a target block geometrically divided into a plurality of sub-blocks according to one aspect of the present disclosure.

[0022] FIG. 13 is a flowchart illustrating a method for encoding residual signals within a target block geometrically divided into a plurality of sub-blocks according to one aspect of the present disclosure.

[0023] FIG. 14 and FIG. 15 are exemplary diagrams for explaining a method of dividing a target block into non-rectangular regions and rearranging residual signals within the non-rectangular regions according to one aspect of the present disclosure.

[0024] Fig. 16 is a flowchart illustrating a decoding method performed by an image decoding device corresponding to the encoding method of Fig. 13.

[0025] FIG. 17 is another exemplary diagram illustrating a method of dividing a target block into non-rectangular regions and rearranging residual signals within the non-rectangular regions according to one aspect of the present disclosure.

[0026] FIG. 18 is an exemplary diagram illustrating a method for determining a quantization parameter according to one aspect of the present disclosure.

[0027] Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. When assigning identification numbers to components in each drawing, it should be noted that, where possible, identical components are assigned the same numbers, even if they appear in different drawings. Furthermore, when describing the present invention, detailed descriptions of known related components or functions will be omitted if they are deemed to obscure the gist of the present invention.

[0028] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to FIG. 1.

[0029] The video encoding device may be configured to include a block division unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0030] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0031] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or / slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced in a sequence composed of multiple pictures is encoded in a Sequence Parameter Set (SPS). In addition, information commonly applied to a single tile or a group of tiles may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile, or tile group header may be referred to as high-level syntax.

[0032] Additionally, the bitstream may include one or more Adaptation Parameter Sets (APS), which contain parameters referenced by a picture or a group of pixels smaller than a picture, e.g., a slice. A picture header or slice header includes an ID for identifying an APS to be used in the corresponding picture or slice. Pictures referencing different PPSs or slices referencing different picture headers may share the same parameters via the same APS ID.

[0033] Each of the multiple pictures may be divided into multiple subpictures that can be independently encoded / decoded and / or displayed. When subpicture division is applied, information regarding the layout of the subpictures within the picture is signaled.

[0034] The block division unit (110) determines the size of the CTU (Coding Tree Unit). Information about the size of the CTU (CTU size) is encoded as a syntax of SPS or PPS and transmitted to the image decoding device.

[0035] The block division unit (110) divides each picture constituting an image into a plurality of Coding Tree Units (CTUs) having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is the basic unit of encoding.

[0036] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0037] Figure 2 shows the QTBTTT split tree structure. As shown in Figure 2, a CTU can first be split into a QT structure. The quadtree splitting can be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, it can be further split into one or more of the BT structure or the TT structure. There can be multiple splitting directions in the BT structure and / or the TT structure. For example, there can be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As shown in Fig. 2, when MTT splitting starts, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has been performed, a flag additionally indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. Alternatively, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer. If the CU split flag (split_cu_flag) value indicates that it is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is the basic unit of encoding.If the CU split flag (split_cu_flag) value indicates that the CU is split, the video encoding device starts encoding from the first flag in the manner described above.

[0038] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0039] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0040] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0041] The intra prediction unit (122) constructs reference samples from previously restored samples located around the current block within the current picture including the current block, and predicts samples within the current block using the reference samples. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3, the multiple intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and 65 directional modes. The reference samples to be used and the calculation formula are defined differently depending on each prediction mode.

[0042] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate rate-distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best rate-distortion characteristics among the tested modes.

[0043] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using reference samples and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0044] The inter prediction unit (124) generates a prediction block for the current block through a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and uses the searched block to generate a prediction block for the current block. Then, a motion vector corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and a motion vector calculated based on the luma component is used for both the luma component and the chroma component. Motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0045] The inter prediction unit (124) may perform interpolation for a reference picture or a reference block to improve the accuracy of prediction. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed for the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision of the motion vector, i.e., the motion vector resolution, can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such an adaptive motion vector resolution is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled.

[0046] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the encoding unit (150). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0047] Motion information (motion vectors, reference pictures) used for inter prediction must be signaled to the video decoding device. Various methods can be used to minimize the number of bits required to encode the motion information.

[0048] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0049] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0050] As the surrounding blocks for deriving merge candidates, all or part of the left block (L), upper block (A), upper right block (AR), lower left block (BL), and upper left block (AL) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the co-located block may be additionally used as a merge candidate.

[0051] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, a merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the encoding unit (155) and transmitted to the decoding device.

[0052] Another method for encoding motion information is AMVP mode.

[0053] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (L), upper block (A), upper right block (AR), lower left block (BL), and upper left block (AL) adjacent to the current block in the current picture shown in FIG. 2 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block located in the same location as the current block (co-located block) in the reference picture or blocks adjacent to the block in the same location may be used.

[0054] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0055] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information for identifying the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0056] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0057] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0058] The transformation unit (140) can transform residual signals within a residual block. The two-dimensional size of the residual block can be used as a transform unit (hereinafter referred to as "TU"), which is a block size for performing transformation. Alternatively, the residual block can be divided into multiple sub-blocks and each sub-block can be used as a TU to transform the residual signals within the corresponding sub-blocks.

[0059] The transform unit (140) divides the residual block into one or more sub-blocks, applies a transform to the one or more sub-blocks, and transforms the residual values ​​of the transform blocks from the pixel domain to the frequency domain. In the frequency domain, the transformed blocks are referred to as coefficient blocks or transform blocks that include one or more transform coefficient values. A two-dimensional transform kernel may be used for the transform, and a one-dimensional transform kernel may be used for each of the horizontal and vertical directions. The transform kernel may be based on a discrete cosine transform (DCT), a discrete sine transform (DST), etc. The transform kernel may also be referred to as a transform matrix.

[0060] The transformation unit (140) can individually perform transformations in the horizontal and vertical directions on the residual block or transformation unit. For the transformation, various types of transformation kernels or transformation matrices can be used. For example, a pair of transformation kernels for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation kernel pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation kernel pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0061] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. A matrix of quantization coefficients applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0062] The reordering unit (150) can reorder coefficient values ​​for quantized residual values. The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence through coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans a two-dimensional coefficient array in the column direction or a horizontal scan that scans two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, a scanning method to be used among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan may be determined.

[0063] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0064] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (in the case of a merge mode, a merge index, and in the case of an AMVP mode, information about a reference picture index and a differential motion vector) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

[0065] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0066] The addition unit (170) adds the restored residual block and the prediction block generated by the prediction unit (120) to restore the current block. Samples within the restored current block are used as reference samples when intra-predicting the next block.

[0067] The loop filter unit (180) performs filtering on restored samples to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include one or more of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186).

[0068] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) performs additional filtering on the deblocking-filtered image. The SAO filter (184) is a filter used to compensate for differences between restored samples and original samples caused by lossy coding, and is performed by adding an offset corresponding to each restored sample. The ALF (186) performs filtering on the target sample by applying filter coefficients to the target sample to be filtered and the surrounding samples of the target sample. The ALF (186) can divide the samples included in the image into a predetermined group and then determine one filter to be applied to the group to differentially perform filtering on each group. Information on the filter coefficients to be used in the ALF can be encoded and signaled to the image decoding device.

[0069] The restored blocks filtered through the loop filter unit (180) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0070] FIG. 5 is an exemplary functional block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0071] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0072] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0073] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information and information on residual signals required to restore the current block.

[0074] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0075] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (MTT_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Through this, each node below the leaf nodes of QT is recursively split into the BT or TT structure.

[0076] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0077] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0078] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded through division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts a syntax element for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts a syntax element for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0079] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about the residual signal.

[0080] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0081] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients using quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0082] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a restored residual block for the current block. In addition, when MTS is applied, the inverse transform unit (530) determines the transform kernel or transform matrix to be applied in the horizontal and vertical directions, respectively, using the MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0083] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0084] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference samples around the current block according to the intra prediction mode.

[0085] The inter prediction unit (544) uses the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0086] The inter prediction unit (544) may also perform interpolation filtering according to the value of the motion vector of the current block. That is, if the fractional part of the motion vector is not 0, the inter prediction unit (544) generates subsamples indicated by the fractional part through interpolation. If the fractional part of the horizontal component (x component) of the motion vector is not 0, interpolation is performed in the horizontal direction, and if the fractional part of the vertical component (y component) of the motion vector is not 0, interpolation is performed in the vertical direction.

[0087] An adder (550) restores the current block by adding the residual block output from the inverse transform unit and the predicted block output from the inter prediction unit or the intra prediction unit. Samples within the restored current block are used as reference samples when intra-predicting a block to be decoded later.

[0088] The loop filter unit (560) may include at least one of a deblocking filter (562), an SAO filter (564), and an ALF (566). The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) performs filtering by adding an offset corresponding to the restored block after deblocking filtering to compensate for differences between restored samples and original samples caused by lossy coding. The ALF (566) applies filter coefficients to a target sample to be filtered and surrounding samples of the target sample to perform filtering on the target sample. The ALF (566) divides samples included in an image into predetermined groups and determines one filter to be applied to each group to perform differential filtering for each group. The filter coefficients of the ALF are determined using information on filter coefficients decoded from the non-stream.

[0089] The restored block filtered through the loop filter unit (560) is stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within the picture to be encoded later.

[0090]

[0091] As described above, conventional video encoding and decoding devices encode and decode pictures in units of square or rectangular blocks. However, if a picture contains edges (e.g., boundaries between objects in a picture) in directions other than horizontal or vertical, such square or rectangular block-based encoding may reduce encoding efficiency. For example, a CTU to be partitioned into a tree structure may contain diagonal edges. In order to efficiently encode the CTU, CUs within the CTU containing the edges must be recursively partitioned into smaller square or rectangular CUs. However, the recursive partitioning rather increases the amount of encoding of partition information related to the tree-structure partitioning of the CTU.

[0092] On the other hand, to reduce the amount of encoding of segmentation information, or due to constraints on the maximum level or depth at which a CTU can be split into a tree structure, a video encoding device may no longer segment CUs containing diagonal lines. Encoding such CUs based on square or rectangular blocks increases the number of bits required to encode the coded data of the CU, i.e., the residual signal.

[0093] The present disclosure provides a method for efficiently encoding pictures containing edges of various directions using an arbitrary form of block partitioning, i.e., geometric block partitioning.

[0094] 1. Block division

[0095] An image encoding device divides a square or rectangular target block into non-rectangular blocks. Here, a rectangular block means a block that is square or rectangular in shape, and a non-rectangular block means a block that is not square or rectangular in shape.

[0096] If an edge exists within the target block, in other words, if the target block includes a boundary between objects, the image encoding device can geometrically divide the target block into a plurality of non-rectangular sub-blocks based on the edge.

[0097] FIG. 6 is an exemplary diagram for geometrically dividing a target block according to one embodiment of the present invention.

[0098] As illustrated in (A) of FIG. 6, if an edge (602) exists within the target block, the image encoding device can divide the target block into subblock A and subblock B based on the edge (602). As illustrated in (B) of FIG. 6, if two edges (612, 614) exist within the target block, the image encoding device can divide the target block into three subblocks A, B, and C.

[0099] An image encoding device encodes segmentation information related to the geometric segmentation of a target block. The segmentation information may include a flag indicating whether geometric segmentation, i.e., block segmentation of any shape, has been applied to the target block. Furthermore, if the flag indicates geometric segmentation, the segmentation information includes segmentation mode information related to the geometric segmentation shape of the target block.

[0100] The segmentation mode information includes information indicating boundaries between subblocks, and may further include information indicating the number of boundaries.

[0101] In one embodiment, information about a boundary line may be defined as the positions or coordinates of both end points where the boundary line meets the boundary of a target block. The positions of both end points may be expressed with various precisions. The accuracy of the positions of both end points of the boundary line may be expressed in units of integer pixels, or may be expressed in units of fractional pixels such as 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, etc. The video encoding device may directly signal the positions of both end points, or alternatively, may signal indices indicating the coordinates of both end points. For example, each side of the target block may be divided into a predetermined number of equal intervals, and an index may be assigned to each of the division points of each side. The video encoding device may encode the index of the division point corresponding to each of the both end points of the boundary line of the target block. The index for the division point may be assigned in various ways. For example, an index may be assigned to each division point in a clockwise direction from the upper left division point of the target block. Alternatively, indices for splitting points can be independently assigned to each edge of the target block. For example, indices starting from 0 can be assigned to each edge of the target block. In this case, the image encoding device encodes a flag indicating whether a splitting point exists on the boundary line for each edge of the target block, and an index for the edge where the splitting point exists.

[0102] In another embodiment, information about a boundary line may be information indicating the angle and position of the boundary line. The position of the boundary line may be expressed as a distance from the center of the target block to the boundary line. The angle and distance of the boundary line may be expressed with various precisions. For example, the angles of the boundary line may be expressed in integer units such as 1 degree, 2 degrees, etc., or may be expressed in fractional units such as 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32, etc. The distance of the boundary line may also be expressed with a precision of integer pixels or subpixels. The video encoding device may directly signal the angle and distance of the boundary line, or alternatively, may signal indices corresponding to the angle and distance of the boundary line within the target block, respectively, among predefined angles and distances.

[0103] The endpoints of the boundary line described above or the angles and distances of the boundary line can be predictively encoded. For example, the video encoding device can select a block that is identical or similar to the partitioning mode of the target block among the pre-encoded blocks, and encode information about the difference between the endpoints of the boundary line of the selected block and the endpoints of the boundary line of the target block, or the difference between the distance and angle of the boundary line of the selected block and the distance and angle of the boundary line of the target block. In addition, selection information indicating which block has been selected among the pre-encoded blocks can be encoded. Here, the pre-encoded blocks can be selected from a predetermined number of blocks that have been recently encoded using geometric partitioning, for example, from neighboring blocks adjacent to the target block, as illustrated in FIG. 4, or alternatively or additionally. Since the video encoding device and the video decoding device can select the pre-encoded blocks in the same manner, signaling of additional information for configuring the pre-encoded blocks is not required.

[0104] As another embodiment of expressing information about a boundary line, the boundary line information can be expressed as an index indicating a boundary line within a target block among boundary lines in various predefined directions. Fig. 7 is an exemplary diagram showing various predefined boundary lines. Fig. 7 shows 64 boundary lines. An image encoding device encodes an index corresponding to a boundary line of a target block among the 64 boundary lines. In a case where segmentation of the target block using multiple boundary lines is allowed, the image encoding device can signal the number of boundary lines and the corresponding number of indices.

[0105] The image encoding device encodes the segmentation information described above and signals it to the image decoding device, and the image decoding device uses the segmentation information to segment the target block into sub-blocks in the same way as the image encoding device.

[0106] Meanwhile, the image encoding device may signal a flag indicating whether geometric segmentation is enabled at a level higher than the target block, such as the level of SPS, PPS, slice, tile, etc. When the flag indicates that geometric segmentation is enabled, the aforementioned segmentation information is signaled for blocks within the image area corresponding to that level.

[0107]

[0108] 2. Prediction

[0109] The geometric partitioning described above can be applied to both inter-prediction encoding and intra-prediction encoding. When a target block is geometrically partitioned into a plurality of sub-blocks, the prediction unit (120) of the video encoding device performs inter-prediction or intra-prediction on each sub-block. The prediction information used for inter- or intra-prediction is signaled to the video decoding device, and the prediction unit (540) of the video decoding device uses the prediction information to predict each sub-block in the same manner as the prediction unit (120) of the video encoding device.

[0110] 2-1) Inter prediction

[0111] The inter prediction unit (124) of the video encoding device determines a motion vector for each sub-block and generates prediction sub-blocks corresponding to each sub-block using the motion vector. Then, the prediction sub-blocks are combined to generate a prediction block for the target block. In order to remove artifacts at the boundaries between sub-blocks, the inter prediction unit (124) may perform additional filtering on prediction pixels near the boundaries between sub-blocks. The motion information for each sub-block is encoded by the encoding unit (155) and transmitted to the video decoding device. The inter prediction unit (544) of the video decoding device generates a prediction block of the target block by predicting each sub-block using the received motion information in the same manner as the inter prediction unit (124) of the video encoding device.

[0112] Meanwhile, geometric segmentation may be applied only when the target block is encoded in merge or skip mode. In this case, the inter prediction unit (124) constructs a merge list including a predetermined number of merge candidates from the surrounding blocks of the target block, as described above. Subsequently, the inter prediction unit (124) selects a merge candidate to be used as a motion vector of each sub-block from among the merge candidates included in the merge list and generates merge index information for identifying the selected candidate.

[0113] The merge index information for each sub-block is encoded by the encoding unit (155) and transmitted to the video decoding device. The inter prediction unit (544) of the video decoding device, which has received the merge index information for each sub-block, constructs a merge list, similar to the inter prediction unit (124) of the video encoding device, and selects a merge candidate indicated by the merge index information from the merge list to predict each sub-block.

[0114] 2-2) Intra prediction

[0115] The intra prediction unit (122) of the video encoding device constructs reference pixels (reference samples) from the restored pixels around the target block and determines an intra prediction mode for each sub-block. Then, each sub-block is predicted from the reference pixels according to the determined intra prediction mode. The intra prediction unit (542) of the video decoding device predicts each sub-block in the same manner as the intra prediction unit (122) of the video encoding device, using information received from the video encoding device.

[0116] Since the operation of the intra prediction unit (542) of the image decoding device is the same as the operation of the intra prediction unit (122) of the image encoding device, the operation of the image encoding device will be mainly described below.

[0117] The reference pixels for each subblock can be configured differently based on one or more of the angle (or direction) of the boundary, the position of the boundary within the target block (or the distance from the center of the target block to the boundary), the position of the subblock relative to the boundary, and the position of the surrounding lines (columns and / or rows of pixels) of the target block used to configure the reference pixels. When the target block is divided based on the boundary between objects, the subblocks are likely to have different texture characteristics because they belong to different objects. On the other hand, blocks located in the same direction with respect to the boundary are likely to have similar texture characteristics. Therefore, the reference pixels for each subblock are configured using the surrounding pixels located in the same direction with respect to the boundary as the corresponding subblock.

[0118] Figure 8 is an example diagram explaining a method of configuring reference pixels for each subblock.

[0119] Reference pixels can be configured using surrounding pixels included in the row adjacent to the upper side and the column adjacent to the left side of the target block. As illustrated in FIG. 8, the reference pixels for subblock A are configured with surrounding pixels (802, 804) included in the upper row or the left column and located in the same direction as subblock A with respect to the boundary line. Similarly, the reference pixels for subblock B are configured with surrounding pixels (806) included in the upper row or the left column and located in the same direction as subblock B with respect to the boundary line. The example of FIG. 8 is about a method for configuring reference pixels when encoding and decoding for blocks is performed in the order from left to right in the horizontal direction and from top to bottom in the vertical direction, such as in the raster scan order. However, other encoding and decoding orders may be used, and depending on the encoding and decoding order, adjacent pixels to the right or bottom of the target block may be used as reference pixels.

[0120] The intra prediction unit (122) can specify the boundary within the target block based on the angle (or direction) of the boundary and the location of the boundary within the target block, and calculate the location (P, Q) where the boundary meets the upper row and the left column. The intra prediction unit (122) determines reference pixels located in the same direction as each sub-block based on the boundary using the locations P and Q.

[0121] As illustrated in Fig. 8, reference pixels may be constructed using surrounding pixels included in the upper row and left column immediately adjacent to the target block, but the present invention is not limited thereto. For example, the intra prediction unit (122) may construct reference pixels using restored surrounding pixels included in other rows or columns that are not immediately adjacent to the target block.

[0122] Figure 9 is another example diagram illustrating a method of configuring reference pixels for each subblock.

[0123] As illustrated in FIG. 9, the intra prediction unit (122) can configure reference pixels using not only the line immediately adjacent to the target block but also the surrounding pixels included in at least one non-adjacent line. For example, the surrounding pixels included in a predetermined number of columns on the left side of the target block and a predetermined number of rows above the target block can be used to configure the reference pixels. The intra prediction unit (122) can select which line among the plurality of lines to use. The intra prediction unit (122) can calculate the position where the selected line meets the boundary line and configure the reference pixels for each sub-block differently using the calculated position. Meanwhile, the index information for the selected line among the plurality of lines is encoded and transmitted to the image decoding device. The image decoding device selects a line for configuring reference pixels among the plurality of lines using the received index information. The index for each line (row and column) can be assigned depending on how many lines it is away from the target block. For example, the first line immediately adjacent to the target block may be assigned an index of 0, the second line an index of 1, and so on. Depending on the embodiment, multiple rows and columns may be used instead of a single row and column. Information about multiple rows and columns may be transmitted through each index, and alternatively, an index may be transmitted for selecting one or more candidates from a list of candidates each consisting of two or more pairs of lines. In addition, the indices of the left and upper lines may be configured differently. For example, the first line may be selected in the left column, and the second line may be selected in the upper row.

[0124] The intra prediction modes used to predict each sub-block may include directional modes and non-directional modes (e.g., DC mode, PLANAR mode), as shown in FIG. 3. The intra prediction mode for each sub-block may be selected based on at least one of the angle (or direction) of the boundary and the positions of the sub-blocks within the target block.

[0125] In some embodiments, the directional modes applicable to each subblock may be restricted based on at least one of the angle (or direction) of the boundary and the positions of the subblocks within the target block relative to the boundary. In other words, only some of the intra prediction modes may be selected from all intra prediction modes for each subblock based on at least one of the angle (or direction) of the boundary and the positions of the subblocks within the target block. As described above, each subblock is predicted using different reference samples. Therefore, the intra prediction unit (122) restricts the directional modes applicable to each subblock so that only the reference samples allowed for each subblock can be used. This restriction reduces the amount of bits required to encode the intra prediction modes of each subblock.

[0126] For example, when the angle (or direction) of the boundary is referred to as boundary_angle, the directional modes can be classified into a set of directional modes having angles that fall within a range between boundary_angle and (boundary_angle + 180 degrees), that is, a range from boundary_angle to (boundary_angle + 180 degrees) counterclockwise, and a set of directional modes having angles that fall within a range other than that. The intra prediction unit (122) applies directional modes belonging to different sets to each sub-block. Referring to the example of FIG. 10, the target block is divided into sub-block A and sub-block B by a 45-degree boundary connecting the upper right and lower left corners of the target block. Sub-block A is predicted using the upper left reference pixel (1002) and the upper and left reference pixels (1004) of the target block, and sub-block B is predicted using the upper right and lower left pixels (1006). Therefore, the directional modes applicable to subblock A are limited to intra prediction modes having an angle within the range of 45 degrees counterclockwise to 225 degrees, and the directional modes applicable to subblock B are limited to intra prediction modes having an angle within the range of 45 degrees clockwise to -135 degrees. For various geometric partitioning shapes as shown in Fig. 8, the directional modes allowed for subblocks A and B can be adaptively changed depending on the angle of the boundary and the position of the subblock relative to the boundary.

[0127] Additionally, the available directional modes may be limited depending on the range of available reference pixels. In the embodiment of FIG. 10, the number of reference pixels (1006) of subblock B may be limited by the horizontal and vertical lengths of the target block including subblock B, and accordingly, directional modes that cannot refer to the reference pixels (1006) are not used as intra prediction modes of subblock B. For example, when the size of the target block in FIG. 10 is N x N, the available reference pixels may be limited to 2N consecutive neighboring samples in the horizontal (row) direction from the sample immediately above the upper-leftmost sample in the target block and 2N neighboring pixels in the vertical (column) direction from the sample immediately to the left of the upper-leftmost sample. In this case, the intra prediction modes allowed for subblock B are limited to -130 to -112.5 degrees and 22.5 to 45 degrees.

[0128] In this embodiment, the intra prediction unit (122) selects intra prediction modes to be applied to each subblock based on the boundary line. Then, it determines the intra prediction mode of the corresponding subblock from among the intra prediction modes selected corresponding to each subblock. Information indicating the determined intra prediction mode is transmitted to the image decoding device. The intra prediction unit (542) of the image decoding device, similar to the intra prediction unit (122) of the image encoding device, selects intra prediction modes for each subblock based on the boundary line. Then, it determines the intra prediction mode of each subblock from among the intra prediction modes selected for the corresponding subblock.

[0129] In another embodiment, the constraint on the intra prediction mode for each sub-block may be performed under specific conditions. The intra prediction unit (122) may check the intra prediction mode of at least one neighboring block including at least a portion of the reference pixels configured for each sub-block, and compare the angle indicated by the intra prediction mode of the neighboring block with the angle of the boundary line to determine whether to restrict the intra prediction modes for the corresponding sub-block. For example, the intra prediction unit (122) may check the intra prediction mode of one or more neighboring blocks including at least one pixel among the reference pixels (1002, 1004) of sub-block A. If there is an intra prediction mode of the neighboring block whose angle is outside the range of 45 degrees to 225 degrees, there is a possibility that the texture characteristics of the two regions divided by the boundary line may not be distinguished from each other. Therefore, the intra prediction unit (122) does not restrict the intra prediction modes for sub-block A. In this case, the reference pixels of subblock A may not be limited to the surrounding pixels indicated by 1002 and 1004 in FIG. 10.

[0130] The intra prediction unit (122) determines intra prediction modes of each subblock among intra predictions applicable to each subblock, and generates prediction subblocks for each subblock using the determined intra prediction modes.

[0131] The intra prediction modes for each subblock are encoded and transmitted to the video decoding device. The video encoding device can directly encode information indicating the intra prediction mode for each subblock, but can also predictively encode it.

[0132] In some embodiments, a video encoding device constructs a candidate list using intra prediction modes of neighboring blocks of a target block. The candidate list is shared for all subblocks. The video encoding device signals index information for identifying the intra prediction mode of each subblock from the candidate list. As an example, the video encoding device selects a prediction mode having a mode number closest to the intra prediction mode of the subblock among the prediction modes included in the candidate list and encodes an index corresponding to the selected prediction mode. Then, the video encoding device encodes a difference value between the actual intra prediction mode of the subblock and the prediction mode corresponding to the encoded index. As another example, the difference value may be encoded if the prediction mode of the subblock is not present in the candidate list, i.e., if the actual intra prediction mode of the subblock and the prediction mode corresponding to the encoded index are not the same. To this end, a flag indicating whether the prediction mode of the subblock is present in the candidate list may be encoded before the difference value.

[0133] In another embodiment, the candidate list may be configured differently for each subblock based on at least one of the direction (or angle) of the boundary line and the position of each subblock divided by the boundary line. Referring to FIG. 10, as described above, subblock A has a high probability of having similar texture characteristics to neighboring blocks that include at least some of the neighboring pixels (1004) located at the top or left of the target block and the neighboring pixels (1002) located at the upper left. Accordingly, the image encoding apparatus may generate a candidate list for subblock A from intra prediction modes of neighboring blocks that include at least some of the neighboring pixels (1002, 1004). Subblock B has a high probability of having similar texture characteristics to neighboring blocks that include at least some of the neighboring pixels (1006) located at the upper right or lower left of the target block. Accordingly, the image encoding apparatus may generate a candidate list for subblock B from intra prediction modes of neighboring blocks that include at least some of the neighboring pixels (1006).

[0134] In the present embodiment, the method for encoding information for identifying the intra prediction mode of the corresponding subblock from each candidate list may be the same as the embodiment described above. As another example, the video encoding device encodes a flag indicating whether the intra prediction mode of the subblock exists in the corresponding candidate list. If the intra prediction mode of the subblock exists in the corresponding candidate list, the device encodes an index corresponding to the intra prediction mode of the subblock in the candidate list. Otherwise, the device encodes identification information for identifying the intra prediction mode of the subblock from among the remaining intra prediction modes excluding the prediction modes included in the candidate list from the intra prediction modes applicable to the subblock.

[0135] The video encoding device can independently and individually encode the intra prediction modes for each subblock, or alternatively, encode the prediction modes for some subblocks in the same manner as described above, and encode the differential values ​​between the intra prediction modes of the remaining subblocks. For example, the video encoding device classifies the subblocks into groups of subblocks having similar prediction modes. Then, for each group, the intra prediction mode of one subblock is encoded using the method of the above-described embodiment, and the intra prediction modes of the remaining subblocks are encoded as differential values ​​between the intra prediction modes of the encoded subblocks. Referring to FIG. 11, considering the boundaries, subblocks A and B, and subblocks C and D are likely to have similar intra prediction modes. Therefore, the video encoding device encodes the intra prediction modes of subblocks A and C according to the above-described embodiments. Then, the prediction modes of subblocks B and D are encoded as the prediction mode difference between subblocks B and A, and the prediction mode difference between subblocks D and C, respectively.

[0136] The intra prediction unit (542) of the video decoding device intra-predicts each sub-block using the intra prediction modes of each sub-block received from the video encoding device. The operation of the intra prediction unit (542) of the video decoding device is identical to the operation of the intra prediction unit (122) of the video encoding device described above, and therefore, further description thereof is omitted.

[0137] Meanwhile, in order to predict each sub-block, matrix-based intra prediction (MIP) can be applied. In order to predict sub-blocks geometrically divided from the target block using the MIP mode, the intra prediction unit (122) configures reference samples for each sub-block in the same manner as described above and performs matrix operations using different reference samples for each sub-block to generate predicted sub-blocks for each sub-block.

[0138] Referring to FIG. 10, the intra prediction unit (122) configures reference samples (1002, 1004) for subblock A and generates an input vector for matrix operation from the reference samples (1002, 1004). The input vector can be generated by downsampling the reference samples. For example, the input vector can be generated by averaging a predetermined number of consecutive reference samples. The intra prediction unit (122) multiplies the input vector by a matrix to generate prediction pixels that configure subblock A. The prediction pixels are rearranged according to the shape of subblock A. The intra prediction unit (122) generates a final prediction subblock corresponding to the shape of subblock A through interpolation using the generated prediction pixels and the reference pixels (1002, 1004) of subblock A.

[0139] The intra prediction unit (122) constructs an input vector from reference samples (1006) for subblock B. Then, in the same manner as subblock A, it generates a prediction subblock corresponding to subblock B.

[0140] Meanwhile, multiple matrices can be used in MIP. Each subblock can use the same matrix or different matrices. Information identifying the used matrix is ​​encoded and transmitted to the image decoding device.

[0141] To predict each sub-block, Intra Block Copy (IBC) may be applied. The intra prediction unit (122) determines block vectors that indicate the block most similar to each sub-block within the decoded area within the current picture including the target block. Then, the decoded pixels of the area indicated by each block vector are used to generate predicted sub-blocks for each sub-block.

[0142] When prediction sub-blocks are generated for each sub-block according to the intra prediction method described above, the prediction sub-blocks are combined to generate a prediction block for the target block. To remove artifacts at the boundaries between sub-blocks, filtering may be performed on prediction pixels near the boundaries between sub-blocks. Alternatively or additionally to the filtering, offsets corresponding to each sample position may be added to prediction samples within the prediction block of the target block. Here, the offsets may be generated through position-based intra prediction. Position-based intra prediction may be the PLANAR mode specified by the VVC standard or may be PDPC (Position Dependent Intra Prediction Combination).

[0143]

[0144] 3. Conversion

[0145] The transform unit (140) of the video encoding device transforms a residual block, which is the difference between a target block and a prediction block of the target block, into a transform coefficient block in the frequency domain. The transform coefficients within the transform coefficient block are encoded and signaled to the video decoding device, and are inversely transformed into a residual block in the spatial domain by the inverse transform unit (530) of the video decoding device.

[0146] In some embodiments of the present disclosure, the transform unit (140) of the video encoding device may transform the target block using a transform unit having the same size as the target block, or may independently transform each sub-block. For example, the transform unit (140) may rearrange residual signals within each sub-block into a rectangular or square shape to generate rectangular or square-shaped residual blocks, and may independently or individually transform the rectangular or square-shaped residual blocks.

[0147] In another embodiment of the present disclosure, as shown in FIG. 12, the transform unit (140) classifies residual signals within a target block into an area of ​​residual signals near a boundary line and an area other than the boundary line, and rearranges the residual signals within each area to generate a rectangular or square residual sub-block. The transform unit (140) individually transforms and quantizes the residual sub-blocks to generate transform coefficient blocks corresponding to each sub-block. The transform coefficients within each transform coefficient block are encoded and transmitted to an image decoding device. The transform unit (530) of the image decoding device inversely transforms and inversely quantizes each of the transform coefficient blocks to generate a plurality of residual sub-blocks. Then, the residual signals within each residual sub-block are arranged to the original positions of the corresponding areas within the target block, thereby generating a residual block for the target block. A method of encoding residual signals within the target block will be described in detail below.

[0148] FIG. 13 is a flowchart illustrating a method for encoding residual signals within a target block geometrically divided into a plurality of sub-blocks by an image encoding device according to one embodiment of the present disclosure.

[0149] The transformation unit (140) of the video encoding device determines a plurality of transformation units to transform the target block (S1110). The transformation unit refers to the size of a one-dimensional or two-dimensional block that is divided from the target block so as not to overlap with each other, and means a unit in which transformation is performed. In addition, the transformation unit (14) can determine a transformation kernel for transforming residual signals corresponding to each transformation unit among the plurality of transformation kernels.

[0150] As illustrated in FIG. 14, the transformation unit (140) divides the target block into a plurality of areas by considering the boundary between the geometrically divided sub-blocks (S1220), and rearranges the residual signals in each area to generate a plurality of rectangular or square residual sub-blocks having the size of the corresponding transformation unit (S1230). The plurality of areas include a boundary area including the boundary line and one or more non-boundary areas not including the boundary line. The number of areas divided from the target block may be the same as the number of transformation units. In addition, the number of samples in each area may be the same as the number of samples in the corresponding transformation unit. For example, if the horizontal length of the transformation unit is 2 m and the vertical length is 2 n When , the number of samples in the area corresponding to the conversion unit is 2 m * 2 n am.

[0151] For example, as illustrated in FIG. 14, the plurality of regions may be a first region (border region) including a boundary line and a second region (non-border region) not including a boundary line in the target block, excluding the first region. The first region may be a set of rows, each consisting of a certain number of horizontally continuous samples. FIG. 14 illustrates four samples in the horizontal direction. However, the present invention is not limited thereto, and the first region may also be a set of columns, each consisting of a certain number of vertically continuous samples. The number of horizontal or vertical samples for forming the first region may be determined in consideration of the size of the transformation unit determined corresponding to the first region, i.e., the number of samples in the transformation unit. The first region is set such that the number of samples in the first region is equal to the number of samples in the corresponding transformation unit. In addition, the type of transformation kernel to be applied to the residual signals in the first region may be additionally considered to determine the number of samples.

[0152] The target block may be divided into three or more regions, rather than just two. For example, as illustrated in Fig. 15, the target block may be divided into three regions (regions 1 to 3) based on the distance from the boundary. The residual signals within each region are rearranged to generate rectangular residual sub-blocks. The size of each region is determined based on the size of the transformation unit corresponding to that region. The number of samples in each region is determined to be equal to the number of samples in the corresponding transformation unit.

[0153] The transformation unit (140) transforms and quantizes each of the rectangular residual sub-blocks to generate a plurality of transformation coefficient blocks (S1240). The transformation information, which includes information about the transformation coefficients within the transformation coefficient blocks and information about the transformation unit, is encoded by the encoding unit (155) and transmitted to the image decoding device. The transformation information may further include information about the transformation kernel.

[0154] Meanwhile, although in the above-described embodiments, all residual sub-blocks corresponding to each region are described as being transformed, quantized, and encoded, only some of the residual sub-blocks may be transformed, quantized, and encoded. The rest are all padded with 0. For example, for a target block predicted through geometric segmentation, only a boundary region including a boundary line among a plurality of regions may be encoded. Alternatively, one or more regions to be transformed, quantized, and encoded among a plurality of regions may be selected, and information representing the selected region may be signaled to an image decoding device.

[0155] Fig. 16 is a flowchart illustrating a decoding method performed by an image decoding device corresponding to the encoding method of Fig. 13.

[0156] The decoding unit (510) of the video decoding device decodes transformation information from a bitstream. Then, based on the transformation information, it determines transformation units in a rectangular or square shape, decodes transformation coefficients corresponding to each transformation unit, and generates transformation coefficient blocks (S1610). The inverse quantization unit (520) of the video decoding device inversely quantizes each transformation coefficient block, and the inverse transform unit (530) inversely transforms the inversely quantized transformation coefficient blocks to generate residual subblocks (S1620).

[0157] The inverse transform unit (530) determines a plurality of regions within the target block by considering the number of transformation units and the size of each transformation unit, the number of transformation units and the size of the boundary specified by the boundary information. That is, a boundary region including the boundary and one or more non-boundary regions not including the boundary within the target block are determined (S1630). In order to identify the plurality of regions, information about the transformation kernel included in the transformation information may be additionally considered in addition to the sizes of the transformation units. The inverse transform unit (530) generates a residual block for the target block by arranging or rearranging the residual signals of each residual sub-block in a corresponding region (S1640).

[0158] As described above, the video encoding device can encode and transmit only a portion of a plurality of regions including a boundary region and one or more non-boundary regions. In this case, the video decoding device decodes, inversely quantizes, and inversely transforms only the transform coefficient blocks corresponding to the portion of the region encoded by the video encoding device to generate a residual sub-block. Then, the residual signals within the residual sub-block are rearranged into the corresponding region. The remaining region is padded with zeros.

[0159] Which area among the multiple areas is encoded may be agreed upon in advance between the video encoding device and the video decoding device, or information indicating the encoded area may be signaled from the video encoding device to the video decoding device.

[0160] In the embodiment of Fig. 12, the image encoding device determines a rectangular or square-shaped transformation unit, then divides the target block into a plurality of regions according to the transformation unit, and rearranges the pixels within each region according to the size of the transformation unit to generate a rectangular residual sub-block. Fig. 16 illustrates a decoding operation corresponding to the encoding operation of Fig. 12.

[0161] As another embodiment, the conversion unit (140) may first divide the target block into multiple regions and then determine the conversion unit.

[0162] The conversion unit (140) divides the target block into a boundary region including a boundary line and one or more non-boundary regions not including a boundary line. Here, how to configure the boundary region and the non-boundary region may be agreed upon in advance between the encoding device and the decoding device depending on the type of the boundary line (e.g., the intersection point where the boundary line meets each side of the target block, or the angle of the boundary line and the position of the boundary line). Alternatively, information related to the region division may be determined at a level higher than the block level, such as the level of a sequence, picture, slice, tile, etc., and signaled to the video decoding device.

[0163] After multiple regions are determined, the transformation unit (140) determines a rectangular or square transformation unit corresponding to each region by considering the total number of pixels in each region or the number of pixels in the horizontal and vertical directions. Then, the residual signals in each region are rearranged according to the size of the corresponding transformation units to generate rectangular or square residual sub-blocks.

[0164] In the present embodiment, additional signaling of transformation information related to the size of the transformation unit may not be required, because the image decoding device can determine the type of the boundary from the information about the boundary and determine the transformation unit from the type of the boundary.

[0165] An image decoding device identifies a boundary type based on information about a boundary and determines rectangular or square transform units based on the boundary type. Then, the transform coefficients corresponding to each transform unit are decoded to generate a plurality of transform coefficient blocks, and the transform coefficient blocks are inversely quantized and inversely transformed to generate residual sub-blocks. The image decoding device identifies a boundary region including a boundary line and one or more non-boundary regions not including a boundary line within a target block, considering the sizes of the boundary line and the transform units. Then, the residual signals within each residual sub-block are rearranged into corresponding regions, thereby generating a residual block of the target block.

[0166] In another embodiment, an image encoding device may set a virtual area outside a target block based on a boundary line, and may set the boundary line area to include not only the area inside the target block but also the virtual area. The residual signals within the virtual area are padded with a fixed specific value or the values ​​of the residual signals within the target block. Referring to FIG. 17, a virtual line parallel to the boundary line is assumed in each of the two areas divided by the boundary line. The area existing between the two virtual lines is set as the boundary line area. In this case, a triangular area formed at the lower left and upper right sides of the target block may be set as the virtual area. The residual signals within the boundary line area including the virtual area are rearranged into a first residual sub-block having a rectangular or square shape. Since the target block is rectangular or square and the boundary line area includes a virtual area outside the target block, the residual signals within the non-boundary area within the target block, excluding the boundary line area, cannot be rearranged into a rectangular or square shape. Accordingly, when the video encoding device rearranges the residual signals within the non-borderline area into a second residual sub-block of a rectangular or square shape, it excludes from the non-borderline area a number of residual signals equal to the number of residual signals within the virtual area. The location of the residual signals within the non-borderline area to be excluded may be agreed upon in advance between the encoding device and the decoding device, or information about the locations from which the residual signals are excluded may be signaled from the encoding device to the decoding device.

[0167] The image decoding device, after restoring the first residual sub-block, rearranges the residual signals within the first residual sub-block into a boundary region, similar to the method described in other embodiments. At this time, the padded virtual region is excluded. Meanwhile, the residual signals within the second residual sub-block are rearranged into a non-boundary region. The residual signals excluded from the non-boundary region by the encoding device are filled with a predicted value. For example, the excluded residual signals may be padded with a specific value, or alternatively, may be padded with an average value of surrounding residual signals. Alternatively, the region from which residual signals are excluded may be filled with restored pixel values ​​surrounding the excluded region at a pixel level after restoration, not at a residual level. The image decoding device first restores pixel values ​​for an area other than the excluded region within the target block. Then, the pixel values ​​of the excluded region are predicted using the restored pixel values ​​surrounding the excluded region. In other words, the image decoding device predicts pixel values ​​of the excluded area so that the sum of residual signals of the excluded area and predicted pixels corresponding to the residual signals is equal to the weighted average value of restored pixel values ​​around the excluded area.

[0168] Below, a method for quantizing or dequantizing a target block is described.

[0169] A plurality of regions (a boundary region and one or more non-boundary regions) within a target block can be quantized or inversely quantized using the same quantization parameter. For example, one quantization parameter is determined for the target block, and each region shares the quantization parameter of the target block. The image encoding device encodes a differential quantization parameter, which is the difference between the quantization parameter of the target block and the prediction quantization parameter. Here, the prediction quantization parameter is generated from the quantization parameters of the restored blocks adjacent to the target block. Referring to FIG. 18, the prediction quantization parameter is generated from the quantization parameters of at least some of the restored blocks among the neighboring blocks "a" to "g". For example, the image encoding device can set the average value of the quantization parameters of the upper block (b) and the left block (c) as the prediction quantization parameter. The image decoding device derives a prediction quantization parameter in the same manner as the image encoding device, and calculates a quantization parameter of the target block by adding the prediction quantization parameter and a differential quantization parameter received from the image encoding device.

[0170] Alternatively, quantization parameters may be independently determined for each region within the target block. The video encoding apparatus signals a differential quantization parameter to be applied to residual signals corresponding to each region. The prediction quantization parameter of each region is predicted using the quantization parameter of a block adjacent to the region. Referring to FIG. 18, the prediction quantization parameter of region A (border region) is predicted from the quantization parameters of at least some of the neighboring blocks "d", "e", and "f". Depending on the embodiment, the value used for quantization parameter prediction may be limited depending on the area of ​​the border region (A) or the area ratio between regions A and B. For example, if the border region (A) is an area 1 to 2 pixels to the left and right of the border as shown in FIG. 17, the quantization parameter is predicted using "d" and "e". In another embodiment, if the border region (A) extends beyond the border region, prediction may be performed using not only "e" and "d" but also "b", "c", "g", and "f". And the predicted quantization parameter may be the average value of the quantization parameters of the restored surrounding blocks among the surrounding blocks. On the other hand, the predicted quantization parameter of region B (non-boundary region) is predicted from the quantization parameters of the restored blocks among the surrounding blocks "a" to "g". For example, it may be the average value of the restored surrounding blocks.

[0171] As another alternative, the video encoding device determines a quantization parameter for one of the plurality of regions (e.g., a boundary region) and derives the quantization parameter for the remaining regions by adding an offset to the quantization parameter. For example, the video encoding device may transmit a differential quantization parameter for the boundary region and omit transmitting the differential quantization parameter for the non-boundary region. In the example of FIG. 18, the non-boundary region (B) may be relatively well predicted compared to the boundary region (A), and thus the absolute magnitude of the residual signals may be smaller. Therefore, for decoding efficiency, the non-boundary region (B) is always quantized with a quantization parameter that adds an offset to the boundary region (A), and transmitting the differential quantization parameter for the non-boundary region (B) may be omitted. The offset may be a positive value added to the quantization parameter of the boundary region (A). The offset may be adaptively determined depending on the prediction mode. Different offsets may be applied depending on whether the prediction mode of the target block is inter prediction or intra prediction. Also, in inter prediction, different offsets may be applied depending on whether the target block is in merge mode or AMVP mode. For example, the offset may have the largest value in merge mode, and the offset may have the smallest value in intra prediction mode. The video decoding device calculates the prediction quantization parameter in the same manner as the video encoding device, and adds the prediction quantization parameter and the differential quantization parameter to determine the quantization parameter of the boundary region. The quantization parameter of the remaining region is derived by adding the same offset as the video encoding device to the quantization parameter of the boundary region. The offset may be a fixed value agreed upon in advance between the video encoding device and the video decoding device, or may be signaled at a high level.

[0172] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0173] Meanwhile, the various functions or methods described in the present disclosure may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0174] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0175] CROSS-REFERENCE TO RELATED APPLICATION

[0176] This patent application claims priority to Korean Patent Application No. 10-2020-0097611, filed in Korea on August 4, 2020, Korean Patent Application No. 10-2020-0099240, filed in Korea on August 7, 2020, and Korean Patent Application No. 10-2021-0102494, filed in Korea on August 4, 2021, all of which are incorporated by reference herein.

Claims

1. In an image decoding method for decoding a target block using intra prediction, A step of decoding boundary information specifying at least one boundary for dividing the target block from a bitstream, the boundary information allowing dividing the target block into a plurality of non-rectangular blocks; A step of determining intra prediction modes for the non-rectangular blocks based on the above boundary information; A step of generating a prediction block of the target block by intra-predicting each of the non-rectangular blocks using the intra-prediction modes; A step of restoring a residual block of the target block from the bitstream; and A step of restoring the target block by adding the prediction block and the residual block. An image decoding method characterized by including:

2. In paragraph 1, The step of determining the intra prediction modes comprises, for each of the non-rectangular blocks, A step of selecting intra prediction modes to be applied to the non-rectangular block based on the direction of the boundary line specified by the boundary line information and the position of the non-rectangular block relative to the boundary line; and A step of determining an intra prediction mode for the non-rectangular block among the selected intra prediction modes using prediction mode information decoded from the bitstream. An image decoding method characterized by including:

3. In paragraph 2, The step of selecting intra prediction modes to be applied to the above non-rectangular blocks is as follows: A step of determining one or more reference locations adjacent to the target block based on the direction and location of the boundary line specified by the boundary line information and the location of the non-rectangular block relative to the boundary line; and A step of constructing a candidate list including intra prediction modes to be applied to the non-rectangular block using intra prediction modes of surrounding blocks including the above reference locations. An image decoding method characterized by including:

4. In paragraph 3, A video decoding method, characterized in that the prediction mode information includes index information for selecting one of the intra prediction modes included in the candidate list, and a difference value between the intra prediction mode of the non-rectangular block and the intra prediction mode indicated by the index information.

5. In paragraph 1, The step of generating a prediction block of the above target block comprises, for each of the non-rectangular blocks, A step of constructing restored reference pixels to be used for predicting the non-rectangular block based on the direction and position of the boundary line specified by the boundary line information and the position of the non-rectangular block relative to the boundary line; and A step of intra-predicting the non-rectangular block using the above reference pixels. An image decoding method characterized by including:

6. In paragraph 1, The step of restoring the above residual block is: A step of restoring a plurality of rectangular transform coefficient blocks from the above bitstream; A step of generating residual sub-blocks by inversely transforming each of the above transformation coefficient blocks; and A step of dividing the target block into a plurality of regions based on a boundary specified by the boundary information, the number of the transform coefficient blocks, and the size of each of the transform coefficient blocks, and restoring the residual block by rearranging the residual signals within each residual sub-block into a corresponding region, An image decoding method, characterized in that the plurality of regions include a boundary region formed near the boundary region and including the boundary line, and one or more non-boundary regions that do not include the boundary line.

7. In an image encoding method for encoding a target block using intra prediction, A step of dividing the target block using at least one boundary line, the boundary line allowing dividing the target block into a plurality of non-rectangular blocks; A step of determining intra prediction modes for the non-rectangular blocks based on the above boundary lines; A step of generating a prediction block of the target block by intra-predicting each of the non-rectangular blocks; A step of generating a residual block of the target block by subtracting the predicted block from the target block; and A step of encoding boundary information for specifying the boundary line and residual signals within the residual block. An image encoding method characterized by including:

8. In paragraph 7, The step of determining the intra prediction modes comprises, for each of the non-rectangular blocks, A step of selecting intra prediction modes to be applied to the non-rectangular block based on the direction of the boundary line and the position of the non-rectangular block relative to the boundary line; and A step of determining an intra prediction mode for the non-rectangular block among the selected intra prediction modes, A video encoding method, characterized in that prediction mode information indicating an intra prediction mode for the above non-rectangular block is encoded.

9. In paragraph 8, The step of selecting intra prediction modes to be applied to the above non-rectangular blocks is as follows: A step of determining one or more reference locations adjacent to the target block based on the direction and location of the boundary line and the location of the non-rectangular block relative to the boundary line; and A step of constructing a candidate list including intra prediction modes to be applied to the non-rectangular block using intra prediction modes of surrounding blocks including the above reference locations. An image encoding method characterized by including:

10. In paragraph 9, A video encoding method, characterized in that the prediction mode information includes index information for selecting one of the intra prediction modes included in the candidate list, and a difference value between the intra prediction mode of the non-rectangular block and the intra prediction mode indicated by the index information.

11. In paragraph 7, The step of generating a prediction block of the above target block comprises, for each of the non-rectangular blocks, A step of constructing restored reference pixels to be used for predicting the non-rectangular block based on the direction and position of the boundary line and the position of the non-rectangular block relative to the boundary line; and A step of intra-predicting the non-rectangular block using the above reference pixels. An image encoding method characterized by including:

12. In paragraph 7, The step of encoding the residual signals within the above residual block is: A step of dividing the target block into a plurality of regions, wherein the plurality of regions include a boundary region formed near the boundary line and including the boundary line, and one or more non-boundary regions not including the boundary line; and For each of the above multiple regions, a step of rearranging residual signals within the region to generate a rectangular residual sub-block and transforming the residual sub-block. An image encoding method characterized by including:

13. In an image decoding device that decodes a target block using intra prediction, A decoding unit that decodes boundary information and transform coefficients specifying at least one boundary line for dividing the target block from a bitstream, the boundary information allowing dividing the target block into a plurality of non-rectangular blocks; A prediction unit that determines intra prediction modes for the non-rectangular blocks based on the boundary information and generates a prediction block of the target block by intra-predicting each of the non-rectangular blocks using the intra prediction modes; An inverse transform unit that restores the residual block of the target block by inversely transforming the above transformation coefficients; and An adder that restores the target block by adding the predicted block and the residual block. An image decoding device characterized by including a .

14. In paragraph 13, The above prediction unit, for each of the non-rectangular blocks, Based on the direction of the boundary line specified by the boundary line information and the position of the non-rectangular block relative to the boundary line, intra prediction modes to be applied to the non-rectangular block are selected, An image decoding device characterized in that it determines an intra prediction mode for the non-rectangular block among the selected intra prediction modes using prediction mode information decoded from the bitstream.

15. In paragraph 13, The above decoding unit restores a plurality of rectangular transform coefficient blocks, each of which includes transform coefficients, from the bitstream, The above inverse transformation part is, Each of the above transformation coefficient blocks is inversely transformed to generate residual sub-blocks, The target block is divided into a plurality of regions based on the boundary specified by the boundary information, the number of the transformation coefficient blocks, and the size of each of the transformation coefficient blocks, and the residual signals in each residual sub-block are rearranged within the corresponding region to restore the residual block. An image decoding device characterized in that the plurality of regions include a boundary region formed near the boundary line and including the boundary line, and one or more non-boundary regions that do not include the boundary line.

Images