Intra prediction method and apparatus using intra prediction mode candidate list

Abstract

Disclosed are an intra prediction method and apparatus using an intra prediction mode candidate list. According to the disclosed method, an intra prediction mode candidate is derived from a reference region indicating a spatial neighboring region or a temporal neighboring region of the current block, a candidate list is generated on the basis of the intra prediction mode candidate, and intra prediction for the current block is performed on the basis of the candidate list.

Description

Intra prediction method and device using an intra prediction mode candidate list

[0001] The present disclosure relates to a video encoding / decoding method, device, and recording medium for storing a bitstream, and more particularly, to an intra prediction method and device using an intra prediction mode candidate list.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] Meanwhile, in intra prediction, the MPM (Most Probable Mode) technique is a method of deriving the intra prediction mode of the current block by using the prediction mode of the surrounding blocks, considering the characteristic that the prediction modes of adjacent blocks are similar.

[0007] However, if the surrounding blocks used to construct the MPM list are not in the intra prediction mode, the prediction mode of the surrounding blocks is immediately set to the planar mode. If the MPM list is constructed using the surrounding blocks set to the planar mode, the optimal prediction mode of the current block may not be included in the MPM list. In other words, if the MPM list is constructed using the surrounding blocks that are not in the intra prediction mode, the encoding / decoding efficiency may be reduced.

[0008] The present disclosure provides a method and device for image encoding or decoding for efficiently encoding or decoding an image and improving objective and subjective image quality of a restored image, and a recording medium for storing a bitstream generated by the image encoding method / device.

[0009] One aspect of the present disclosure provides an image decoding method, comprising: a step of determining a reference region of a current block, performed by an image decoding device; a step of deriving an intra prediction mode candidate of the current block based on information of the reference region; a step of constructing a candidate list of intra prediction modes based on the intra prediction mode candidate of the current block; and a step of performing intra prediction on the current block based on the candidate list.

[0010] One aspect of the present disclosure provides a video encoding method, comprising: a step of determining a reference region of a current block, performed by an video encoding device; a step of deriving an intra prediction mode candidate of the current block based on information of the reference region; a step of constructing a candidate list of intra prediction modes based on the intra prediction mode candidate of the current block; and a step of performing intra prediction on the current block based on the candidate list.

[0011] One aspect of the present disclosure provides a non-transitory computer-readable recording medium storing a bitstream generated by an image encoding device. The bitstream is generated by an image encoding method. The image encoding method includes the steps of: determining a reference region of a current block; deriving an intra prediction mode candidate of the current block based on information of the reference region; constructing a candidate list of intra prediction modes based on the intra prediction mode candidate of the current block; and performing intra prediction on the current block using the candidate list.

[0012] One aspect of the present disclosure provides a method comprising: generating a bitstream for an image, the method being performed by at least one processor; and transmitting data including the bitstream. The step of generating the bitstream includes: determining a reference region of a current block; deriving an intra prediction mode candidate of the current block based on information of the reference region; constructing a candidate list of intra prediction modes based on the intra prediction mode candidate of the current block; and performing intra prediction on the current block using the candidate list.

[0013] According to the present disclosure, an image encoding / decoding device can efficiently encode / decode an image and improve the objective / subjective image quality of a restored image by constructing an MPM list using an intra prediction mode obtained from a reference region indicated by a reference vector of a surrounding block for prediction of a current block.

[0014] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0015] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0016] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0017] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0018] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0019] FIG. 6 is a diagram schematically illustrating the generation of an MPM list according to one embodiment of the present disclosure.

[0020] FIG. 7 is a diagram illustrating the generation of an MPM list according to one embodiment of the present disclosure.

[0021] FIG. 8a, FIG. 8b, FIG. 8c, and FIG. 8d are drawings for explaining the setting of a reference area according to embodiments of the present disclosure.

[0022] FIG. 9a, FIG. 9b, FIG. 9c, and FIG. 9d are drawings for explaining adjustment of a reference area according to embodiments of the present disclosure.

[0023] FIG. 10a, FIG. 10b and FIG. 10c are diagrams for explaining the derivation of a reference intra prediction mode from a reference region according to one embodiment of the present disclosure.

[0024] FIG. 11 is a flowchart of an image encoding method according to one embodiment of the present disclosure.

[0025] FIG. 12 is a flowchart of an image decoding method according to one embodiment of the present disclosure.

[0026] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0027] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0028] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0029] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0030] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0031] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0032] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0033] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0034] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0035] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0036] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0037] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0038] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0039] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0040] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0041] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0042] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0043] The intra prediction unit (122) can determine the intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select the intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0044] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0045] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0046] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0047] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0048] Various methods can be used to minimize the number of bits required to encode motion information.

[0049] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode.'

[0050] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0051] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0052] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0053] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0054] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0055] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0056] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0057] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0058] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0059] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0060] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0061] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0062] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0063] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0064] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0065] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0066] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0067] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

[0068] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0069] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0070] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0071] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0072] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0073] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0074] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0075] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0076] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0077] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0078] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0079] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0080] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0081] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0082] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0083] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0084] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0085] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0086] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0087] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0088] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0089] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0090] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0091] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0092] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0093] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0094] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0095] Hereinafter, an improved coding tool performed by the aforementioned image encoding device or image decoding device is disclosed.

[0096] In the present disclosure, an image processing device refers to at least one of an image encoding device or an image decoding device.

[0097] In intra prediction, the MPM (Most Probable Mode) technique is a method of deriving the intra prediction mode of the current block by using the prediction mode of the surrounding blocks, considering the characteristic that the prediction modes of adjacent blocks are similar.

[0098] MPM technology includes the construction of MPM lists and signaling of MPM information.

[0099] Specifically, the image processing device generates an MPM list including MPM candidates based on intra prediction modes derived from predefined locations spatially adjacent to the current block and statistically frequently used prediction modes.

[0100] Referring to FIG. 4, pixels of neighboring blocks used to derive the intra prediction mode of the current block are illustrated. The prediction modes of neighboring blocks including pixels A0, A1, B0, B1, and B2 can be used as MPM candidates for deriving the prediction mode of the current block. For example, an MPM list can be constructed based on the prediction modes of all neighboring blocks including pixels A0, A1, B0, B1, and B2. Here, pixels A0, A1, B0, B1, and B2 can be referred to as anchor pixels or base pixels, and the neighboring blocks can be referred to as anchor blocks or base blocks.

[0101] The MPM list includes prediction modes and peripheral modes of neighboring blocks that are likely to have pixel characteristics similar to those of the current block as MPM candidates. The MPM list may include the prediction mode of at least one neighboring block among a neighboring block including an A0 pixel, a neighboring block including an A1 pixel, a neighboring block including a B0 pixel, a neighboring block including a B1 pixel, and a neighboring block including a B2 pixel, and may be configured to include peripheral modes of the prediction modes of the neighboring blocks. The peripheral modes may indicate ±1, ±2, or ±3 of the prediction modes. Alternatively, the MPM list may be configured to include a minimum mode among the prediction modes of the neighboring blocks, peripheral modes of the minimum mode, a maximum mode among the prediction modes of the neighboring blocks, peripheral modes of the maximum mode, or a combination thereof. Alternatively, since a planar mode is mainly used in image compression, the MPM list may include a planar mode. Alternatively, the MPM list may be configured to further include predefined directional prediction modes.

[0102] An MPM list may include a primary MPM list and a secondary MPM list. For example, the primary MPM list may include six MPM candidates, and the secondary MPM list may include 16 MPM candidates.

[0103] Once the MPM list is configured, the video encoding device encodes and signals the prediction mode of the current block using the MPM list. Specifically, the video encoding device can encode and transmit a flag indicating whether the prediction mode of the current block is derived from the prediction mode of a neighboring block, and an MPM candidate index indicating the prediction mode of the current block within the MPM list.

[0104] Encoding and decoding the prediction mode of the current block using the MPM list is described in detail in FIGS. 10 and 11.

[0105] An image processing device can improve encoding efficiency of an intra prediction mode by signaling a flag for an MPM and an index for a small number of MPM candidates instead of an index for identifying one of many prediction modes.

[0106] Meanwhile, in certain cases, the prediction mode of the surrounding blocks used in constructing the MPM list may be replaced with a planar mode or a predefined directional mode. For example, if the surrounding block is not in the intra prediction mode, if the surrounding block does not exist, or if the surrounding block cannot be referenced, the prediction mode of the surrounding block may be set to the planar mode. Here, if the surrounding block is not in the intra prediction mode, this may refer to a case where the prediction mode of the surrounding block is one of the Intra Block Copy (IBC) mode using a block vector, the Palette mode, the Matrix-based Intra Prediction (MIP) mode, or the Inter prediction mode using a motion vector. If the surrounding block cannot be referenced, this may refer to a case where the current block is located at the top of the CTU, or at the boundary of the image such as a tile, slice, subpicture, or picture.

[0107] Alternatively, if the surrounding block is BDPCM, the prediction mode of the surrounding block can be replaced with a horizontal mode or a vertical mode based on the direction of the BDPCM (Block-based Delta Pulse Code Modulation) mode.

[0108] However, since the optimal prediction mode of the current block may not be the planar mode or a predefined directional mode, setting the prediction mode of the neighboring blocks only to the planar mode may reduce the encoding / decoding efficiency for the MPM and deteriorate the image quality. For example, when the neighboring blocks are in the IBC mode predicted by referring to other blocks in the same picture or in the inter prediction mode predicted by referring to blocks in other pictures, the MPM list constructed using the prediction modes of the neighboring blocks replaced with the planar mode may not include the optimal prediction mode of the current block, which may reduce the encoding / decoding efficiency and degrade the image quality.

[0109] In the present disclosure, rather than directly setting the prediction mode of a surrounding block to a planar mode, an intra prediction mode is derived based on a reference relationship between a surrounding block and another block, a prediction mode candidate list of a current block is constructed using the derived intra prediction mode, and intra prediction for the current block is performed using the candidate list.

[0110] For example, when a surrounding block is predicted using a block vector or a motion vector, a reference region is determined using the block vector or motion vector used for prediction of the surrounding block, and a list of prediction mode candidates for the current block can be generated using the intra prediction mode obtained from the reference region.

[0111] FIG. 6 is a diagram schematically illustrating the generation of an MPM list according to one embodiment of the present disclosure.

[0112] In Fig. 6, the N1 block including the L pixel at the lower left and the N5 block including the A pixel at the upper right are used as neighboring blocks for constructing the MPM list of the current block (610). The N5 block is predicted by the 30th directional prediction mode, and the N1 block is predicted based on a block indicated by a reference vector relative to the N1 block. It is assumed that the optimal prediction mode of the current block (610) is the 39th directional prediction mode.

[0113] If the prediction mode of the N1 block is replaced with the planar mode for constructing the MPM list, and the MPM list is constructed based on the planar mode of the N1 block and the 30th directional prediction mode of the N5 block, the MPM list may not include the 39th directional prediction mode. If the MPM list cannot be used to encode the 39th directional prediction mode of the current block (610), more bits may be required to encode and signal the 39th directional prediction mode.

[0114] On the other hand, according to an embodiment of the present disclosure, an MPM list including 39 directional prediction modes can be generated by using the reference vector of the N1 block.

[0115] Specifically, a reference region (620) is established by applying the reference vector used to predict the N1 block to the current block (610). Based on the information of the reference region (620), the 39th directional prediction mode is obtained. Based on the 39th directional prediction mode, an MPM list of the current block (610) is generated, and the MPM list may include the 39th directional prediction mode.

[0116] The video encoding device can signal the MPM index corresponding to the directional prediction mode 39 to the video decoding device. By encoding the MPM index rather than encoding the directional prediction mode 39 itself, the number of bits required to encode / decode and signal the prediction mode of the current block (610) can be reduced.

[0117] FIG. 7 is a diagram illustrating the generation of a candidate list according to one embodiment of the present disclosure.

[0118] The generation of a candidate list according to an embodiment of the present disclosure can be performed in each of an image encoding device and an image decoding device.

[0119] In the present disclosure, the reference relationship between a block used for prediction of a neighboring block and the neighboring blocks can be expressed by a reference vector. The reference vector can be defined by its size and direction. If the neighboring block is intra-predicted, the term 'reference vector' can be used interchangeably with a block vector (BV), and if the neighboring block is inter-predicted, the term 'reference vector' can be used interchangeably with a motion vector (MV).

[0120] In the present disclosure, the surrounding blocks in the current picture, the corresponding blocks in the reference picture, and the corresponding surrounding blocks are blocks that were previously restored before the current block.

[0121] Hereinafter, the candidate list of intra prediction modes may refer to the MPM list. The intra prediction mode candidates may be referred to as MPM candidates.

[0122] In FIG. 7, the generation of a candidate list of intra prediction modes includes a step of determining a reference region of a current block (S710), a step of adjusting the reference region (S720), a step of deriving an intra prediction mode candidate of the current block based on information of the reference region (S730), and a step of configuring an intra prediction mode candidate list based on the intra prediction mode candidate (S740).

[0123] In another embodiment, the step of adjusting the reference area (S720) may be skipped.

[0124] In step S710, the image processing device determines a reference area of ​​the current block.

[0125] A reference region is an area whose image characteristics are expected to be identical or similar to those of the current block. The reference region may be the spatial or temporal surrounding area of ​​the current block.

[0126] A reference region can be determined using at least one reference vector. Here, the reference vector is intended to indicate a region used for predicting surrounding blocks of the current block.

[0127] In one embodiment, the reference area may be an area indicated by a reference vector in a reference picture or a current picture. Specifically, the image processing device may determine that the reference area is an area indicated by a reference vector relative to one of the current block in the current picture or a corresponding block of the current block in the reference picture. In other words, the reference area may be an area located at a position away from the current block or the corresponding block according to the size and direction of the reference vector and having the same size as the current block.

[0128] Here, the corresponding block is a co-located block of the current block. A co-located block refers to a block within a reference picture that is in the same relative position as the current block within the current picture. The corresponding block of a neighboring block of the current block refers to a block within the reference picture that has the same relative position as the neighboring block of the current block.

[0129] When a neighboring block is unidirectionally intra-predicted using a single reference vector, the image processing device can determine that the area indicated by the block vector in relation to the current block in the current picture is the reference area of ​​the current block. In other words, the reference area may be an area located at a position shifted by the block vector from the current block.

[0130] When a surrounding block is unidirectionally inter-predicted using a single motion vector, the image processing device can determine that the area indicated by the motion vector is the reference area of ​​the current block based on a corresponding block in a reference picture different from the current picture.

[0131] In one embodiment, the reference region may be any one of the entire region or the overlapping region of candidate regions indicated by a plurality of reference vectors in the reference pictures or the current picture. Specifically, the image processing device may determine that the reference region of the current block is any one of the overlapping region or the entire region of candidate regions indicated by a plurality of reference vectors relative to one of the corresponding blocks of the current block in the reference pictures or the current block in the current picture.

[0132] When a neighboring block is bidirectionally intra-predicted using multiple block vectors, an image processing device can determine a reference region of the current block based on candidate regions indicated by the block vectors relative to the current block within the current picture. For example, the reference region may be the entire region within the current picture, including a first candidate region indicated by one block vector and a second candidate region indicated by another block vector.

[0133] When a neighboring block is bidirectionally inter-predicted using multiple motion vectors, an image processing device can determine a reference region of a current block based on candidate regions indicated by the motion vectors with reference to corresponding blocks of the current block in reference pictures. For example, the reference region may be the entire region of an L0 candidate region indicated by a motion vector in the L0 direction in an L0 reference picture and an L1 candidate region indicated by a motion vector in the L1 direction in an L1 reference picture.

[0134] In step S720, the image processing device can adjust the reference area of ​​the current block. Adjusting the reference area includes selecting, moving, rotating, and scaling the reference area.

[0135] In one embodiment, the image processing device can adjust the reference area of ​​the current block based on an offset. The reference area of ​​the current block can be moved or rotated by a predefined offset.

[0136] An offset for adjusting the reference area can be predefined. For example, the offset can be (+1, +1) or (-1, -1). As another example, the offset can be an angle of 45 degrees counterclockwise from the horizontal axis.

[0137] Alternatively, the offset for adjusting the reference region can be adaptively determined based on the reference vector. For example, the offset can be obtained by scaling or rotating the reference vector. As another example, when the neighboring block is inter-predicted, a scale ratio or an angle value is determined based on the POC (Picture Order Count) difference between the current picture and the reference picture, and the offset can be obtained by scaling or rotating the reference vector based on the determined scale ratio or angle value. For example, if the POC difference is greater than a threshold, the offset can be obtained by increasing the size of the reference vector. As another example, if the POC difference is greater than a threshold, the offset can be obtained by decreasing the size of the reference vector. The scale ratio or angle value can be proportional or inversely proportional to the POC difference.

[0138] Meanwhile, in bidirectional prediction, when two candidate regions are established based on bidirectional reference vectors, the same offset or different offsets can be applied to the two candidate regions. For example, a single predefined offset can be applied to the two candidate regions. Alternatively, different predefined offsets can be applied to the two candidate regions.

[0139] In one embodiment, the image processing device can adjust the reference area of ​​the current block based on the scale ratio. The reference area of ​​the current block can be expanded or reduced by the scale ratio.

[0140] The scale factor for adjusting the reference area can be predefined. For example, the scale factor can be 0.5 or 1.5. The reference area can be reduced by a factor of 0.5 or expanded by a factor of 1.5.

[0141] Alternatively, the scaling ratio for adjusting the reference region can be adaptively determined based on the POC between the current picture and the reference picture. The scaling ratio can be proportional or inversely proportional to the POC difference. Sections according to the POC difference and a scaling ratio corresponding to each section can be defined. For example, when the POC difference is 2 or less, the scaling ratio can be set to 0.5, and when the POC difference is greater than 2, the scaling ratio can be set to 0.25.

[0142] Meanwhile, in bidirectional prediction, when two candidate regions are established based on bidirectional reference vectors, the same scale ratio or different scale ratios can be applied to the two candidate regions. For example, a single predefined scale ratio can be applied to the two candidate regions. Alternatively, different predefined scale ratios can be applied to the two candidate regions.

[0143] In one embodiment, when a surrounding block is subjected to bidirectional intra prediction or bidirectional inter prediction, the image processing device can adjust the reference region by selecting the reference region of the current block from candidate regions indicated by a plurality of reference vectors.

[0144] An image processing device can select a reference region of a current block from candidate regions based on weights of regions indicated by a plurality of reference vectors. Specifically, a neighboring block can be predicted by weighting regions indicated by a plurality of reference vectors based on blocks corresponding to neighboring blocks of the current block in reference pictures, or by weighting regions indicated by a plurality of reference vectors based on the neighboring block in the current picture. Here, a block corresponding to a neighboring block refers to a block having the same relative position as a neighboring block of the current block among blocks in the reference picture. The image processing device can select a portion of a region based on the magnitudes of the weights of regions used for prediction of the neighboring block, and select a candidate region indicated by using a reference vector indicating the selected region among candidate regions of the current block as a reference region of the current block.

[0145] Alternatively, the image processing device may select a reference region of the current block from among the candidate regions based on the number of sub-blocks within each candidate region. For example, the image processing device may determine that the reference region is a candidate region that includes a small number of sub-blocks among the candidate regions. As another example, the image processing device may determine that the reference region is a candidate region that includes a large number of sub-blocks among the candidate regions.

[0146] In this way, the image processing device can select a reference region from candidate regions indicated by a plurality of reference vectors.

[0147] The image processing device may adjust the reference area of ​​the current block multiple times. More than one adjustment, including selection, translation, rotation, or scaling, may be applied to the reference area.

[0148] Adjustment of the reference region can be performed based on motion information according to the reference vector and information about the reference region. The motion information according to the reference vector can include spatial motion displacement, information about the reference picture, index information about the reference picture, weight information for bidirectional prediction, or index information indicating a weighted sum. Information about the reference region can include block partitioning information within the reference region, prediction mode information, or POC (Picture Order Count) information.

[0149] In step S730, the image processing device derives an intra prediction mode candidate based on information of the reference area.

[0150] The image processing device may determine an intra prediction mode candidate of the current block based on an intra prediction mode of at least one sub-block including a pixel at a preset position among the sub-blocks within a reference region of the current block. Specifically, the image processing device may select at least one block among the sub-blocks within the reference region of the current block based on whether the sub-blocks include a pixel at a preset position, and determine the prediction mode of the selected sub-block as an intra prediction mode candidate of the current block. The preset position may refer to any position within the reference region, or one or more of the upper left, upper right, center, lower left, or lower right within the reference region.

[0151] An image processing device can determine an intra prediction mode candidate of a current block based on areas and prediction modes of subblocks within a reference region. Specifically, the image processing device can select at least one block among the subblocks based on the areas of the subblocks. The area of ​​the subblock can be defined by the number of pixels in the subblock or the product of the width and the height. The image processing device can select a subblock having a larger area among the subblocks. Alternatively, the image processing device can select a subblock having a smaller area among the subblocks. The image processing device can determine the prediction mode of the selected subblock as an intra prediction mode candidate of the current block.

[0152] An image processing device may determine an intra prediction mode candidate of a current block based on the shapes and prediction modes of subblocks within a reference region. Specifically, the image processing device may select at least one block among the subblocks based on the shapes of the subblocks. The shape may be defined by the width-to-height ratio of the subblock, or whether it is square / rectangular. The image processing device may select a subblock that is most similar to the shape of the current block. The image processing device may determine the prediction mode of the selected subblock as an intra prediction mode candidate of the current block.

[0153] Alternatively, the image processing device may obtain gradient information of the reference region by applying a predetermined filter to the reference region of the current block, and determine an intra prediction mode candidate using the gradient information of the reference region. Here, the predetermined filter may be an edge detection filter. The gradient information may indicate directional characteristics of pixel values ​​within the reference region and may include gradient magnitude information and gradient direction information. The image processing device may calculate an accumulated gradient magnitude for each directional prediction mode based on the gradient information of the reference region, and determine an intra prediction mode candidate from the directional prediction modes based on the accumulated gradient magnitude.

[0154] In step S740, the image processing device configures an intra prediction mode candidate list based on the intra prediction mode candidates.

[0155] An image processing device can generate a candidate list based on the prediction mode of a neighboring block and add an intra prediction mode candidate to the candidate list. For example, if a neighboring block is predicted by a reference vector, the image processing device can replace the prediction mode of the neighboring block with a planar mode, generate a candidate list based on the replaced planar mode of the neighboring block and the intra prediction mode of another neighboring block, and add the intra prediction mode candidate to the candidate list. When the number of intra prediction mode candidates in the candidate list is 5, the candidate list can include 6 intra prediction mode candidates by adding the intra prediction mode candidate to the candidate list. The intra prediction mode candidate can be added to the candidate list as the first intra prediction mode candidate or as the last intra prediction mode candidate.

[0156] Alternatively, the image processing device may generate a candidate list by replacing the prediction mode of a neighboring block with an intra prediction mode candidate and constructing the candidate list based on the replaced intra prediction mode of the neighboring block and the intra prediction mode of another neighboring block. If there is a neighboring block predicted using a reference vector, the prediction mode of the neighboring block may be replaced with the derived intra prediction mode instead of the planar mode. For example, when the lower left block of the current block is inter-predicted, the upper right block is intra-predicted 30 times, and the intra prediction mode candidate is intra prediction mode 50, the prediction mode of the lower left block may be replaced with intra prediction mode 50, and an intra prediction mode candidate list may be constructed based on intra prediction mode 50 and intra prediction mode 30. The candidate list may include six intra prediction mode candidates.

[0157] FIG. 8a, FIG. 8b, FIG. 8c, and FIG. 8d are drawings for explaining the setting of a reference area according to embodiments of the present disclosure.

[0158] In FIGS. 8A, 8B, 8C, and 8D, the surrounding block (813) is the lower left block of the current block (811). In another embodiment, in addition to the lower left block, the reference area of ​​the current block may be set using the reference vector of at least one surrounding block described in FIG. 4.

[0159] In Fig. 8a, the surrounding block (813) is a unidirectional block vector BV within the current picture. L It is predicted by referring to the block pointed to. Here, BV is based on the current block (811) in the current picture. L The area indicated by (815) can be set as a reference area.

[0160] In Fig. 8b, the surrounding block (813) is the bidirectional block vectors BV0 within the current picture. L and BV1 LIt is predicted by referring to the blocks pointed to. Specifically, the surrounding blocks (813) are block vectors BV0 L and BV1 L It is predicted by weighting the areas indicated by .

[0161] Here, BV0 is based on the current block (811) in the current picture. L Candidate area 0 (817) indicated by and BV1 based on the current block (811) L At least one of the candidate regions 1 (819) indicated by may be determined as the reference region of the current block (811). Alternatively, the entire region or overlapping region of the two candidate regions (817, 819) may be set as the reference region of the current block (811).

[0162] In Fig. 8c, the corresponding block (821) and the corresponding surrounding block (823) in the reference picture are blocks having the same relative positions as the current block (811) and the surrounding block (813) in the current picture. The surrounding block (813) is a unidirectional motion vector MV from the corresponding surrounding block (823) in the reference picture. L It can be predicted by referring to the area at a distance away.

[0163] Here, MV from the corresponding block (821) on the reference picture L An area (825) located at a distance can be determined to be the reference area of ​​the current block (811).

[0164] In Fig. 8d, reference picture 0 is a reference picture in the L0 direction, and reference picture 1 is a reference picture in the L1 direction. Corresponding block 0 (831) and corresponding neighboring block 0 (833) in reference picture 0 are blocks having the same relative positions as the current block (811) and neighboring block (813), and corresponding block 1 (841) and corresponding neighboring block 1 (843) in reference picture 1 are blocks having the same relative positions as the current block (811) and neighboring block (813). The neighboring block (813) is a block having an L0 motion vector MV0 from corresponding neighboring block 0 (833) in reference picture 0.L The area at a distance of MV1, and the corresponding surrounding block 1 (843) in the reference picture 1, the L1 motion vector MV1 L It can be predicted by referring to the area at a distance of that distance. The surrounding block (813) can be predicted by weighting the two areas.

[0165] Here, MV0 is based on corresponding block 0 (831) in reference picture 0. L MV1 based on candidate region 0 (835) indicated by and corresponding block 1 (841) in reference picture 1 L At least one of the candidate areas 1 (845) indicated by may be set as a reference area of ​​the current block (811).

[0166] In another embodiment, MV0 is based on the current block (811) in the current picture. L Candidate area indicated by , and MV1 based on the current block (811) L At least one of the candidate areas indicated by may be set as a reference area of ​​the current block (811). The entire area or overlapping area of ​​the two candidate areas may be set as a reference area of ​​the current block (811).

[0167] FIG. 9a, FIG. 9b, FIG. 9c, and FIG. 9d are drawings for explaining adjustment of a reference area according to embodiments of the present disclosure.

[0168] In Fig. 9a, the surrounding blocks (913) of the current block (911) are two motion vectors MV0 L and MV1 L Inter prediction is performed by bidirectional prediction using . The reference region of the current block (911) is MV0 within the reference pictures. L and MV1 L It may be an area selected from candidate areas (925, 935) based on the weights of the areas indicated by .

[0169] Specifically, the corresponding neighboring block 0 (923) and the corresponding neighboring block 1 (933) within the reference picture 0 and the reference picture 1 are blocks having the same relative position as the neighboring block (913) and are used to predict the neighboring block (913). The neighboring block (913) is MV0 from the corresponding neighboring block 0 (923) within the reference picture 0. L MV1 from the corresponding peripheral block 1 (933) within the reference picture 1 and the area located at a distance of L By weighting the areas located at a distance of MV0, it can be predicted. L Weight w of the area indicated by MV0 is 0.75, and MV1 L Weight w of the area indicated by MV1 is 0.25.

[0170] Here, motion vector MV0 L Since the candidate area 0 (925) established based on the motion vector MV0 is likely to have similar pixel characteristics to the current block (911), L Candidate area 0 (925) set based on can be selected as the reference area of ​​the current block (911). Or, motion vector MV1 L Reference area candidate 1 (935) set based on may be selected as the reference area of ​​the current block (911).

[0171] In another embodiment, the surrounding blocks (913) of the current block (911) are two block vectors BV0 L and BV1 L When bidirectional intra prediction is performed using , the reference region of the current block (911) can be selected from candidate regions based on the weights of the regions indicated by the two block vectors.

[0172] In Fig. 9b, the surrounding blocks (913) of the current block (911) are two motion vectors MV0 L and MV1 LInter prediction is performed by bidirectional prediction using . The reference region of the current block (911) may be a region selected from the candidate regions (925, 935) of the current block (911) based on the number of sub-blocks included in each of the candidate regions (925, 935).

[0173] Specifically, candidate region 0 (925) in reference picture 0 is divided into 6 sub-blocks, and candidate region 1 (935) in reference picture 1 is divided into 9 sub-blocks. In other words, candidate region 0 (925) includes 6 sub-blocks, and candidate region 1 (935) includes 9 sub-blocks.

[0174] Here, candidate region 0 (925), which includes a smaller number of sub-blocks than candidate region 1 (935), is more likely to be predicted by the same mode as the optimal prediction mode of the current block (911), so candidate region 0 (925) can be selected as the reference region of the current block (911).

[0175] Alternatively, candidate region 1 (935) is predicted by more prediction modes than candidate region 0 (925), and thus is likely to include a block predicted by the same mode as the optimal prediction mode of the current block (911). Therefore, candidate region 1 (935) may be selected as the reference region of the current block (911).

[0176] In another embodiment, the surrounding blocks (913) of the current block (911) are two block vectors BV0 L and BV1 L In the case of bidirectional intra prediction using, one of the reference region candidates can be selected as the reference region of the current block (911) based on the number of sub-blocks included in each of the candidate regions according to the block vectors.

[0177] In Fig. 9c, the surrounding block (913) of the current block (911) is a unidirectional motion vector MV L Inter prediction is performed using the corresponding block (941) and MV LAn initial reference area (945) at a position that is a distance away can be set. The initial reference area (945) of the current block (911) can be adjusted by moving it by a predefined offset. The adjusted reference area (947) is determined to be the final reference area of ​​the current block (911).

[0178] In another embodiment, the offset may be applied to an initial reference region established based on any one of unidirectional block vectors, bidirectional block vectors, or bidirectional motion vectors.

[0179] In FIG. 9d, the initial reference area (945) can be scaled, and the scaled reference area (949) is determined to be the reference area of ​​the current block (911). Here, the scale ratio or scale factor can be a predefined value. For example, the scale ratio can be 0.5 or 1.5.

[0180] FIG. 10a, FIG. 10b and FIG. 10c are diagrams for explaining the derivation of an intra prediction mode candidate from a reference region according to one embodiment of the present disclosure.

[0181] To construct a candidate list, an intra prediction mode candidate is derived based on at least one of a subblock to which a pixel at a preset location within the reference region belongs, a subblock having a large area within the reference region, a subblock having a shape similar to that of the current block, or a filtering result of the reference region.

[0182] In Fig. 10a, intra prediction mode candidates are derived from subblocks to which pixels at predetermined locations within the reference area (1013) belong.

[0183] Specifically, the reference region (1013) of the current block (1011) includes five sub-blocks predicted by intra prediction modes 20, 21, 51, 18, and 50. The sub-block to which the top-left (TL) pixel belongs is predicted by intra prediction mode 20, the sub-block to which the top-right (TR) pixel belongs is predicted by intra prediction mode 51, the sub-blocks to which the center pixel and the bottom-right (BR) pixel belong are predicted by intra prediction mode 50, and the sub-block to which the bottom-left (BL) pixel belongs is predicted by intra prediction mode 18.

[0184] If the pixel at the preset position for constructing the candidate list is a TR pixel, the 51st intra prediction mode is determined to be the intra prediction mode candidate.

[0185] If the pixel at the preset position for constructing the candidate list is a TL pixel, the 20th intra prediction mode is determined to be the intra prediction mode candidate.

[0186] In Fig. 10b, an intra prediction mode candidate is derived by applying a predetermined filter to a reference region (1023). The predetermined filter may be a 3x3 sized edge detection filter.

[0187] Specifically, by applying a 3x3 sized edge detection filter to the reference area (1023), the magnitude and direction of the gradient within the 3x3 area are calculated. The gradient magnitude is accumulated in a bin representing the directional prediction mode closest to the direction of the gradient. The edge detection filter is applied to all areas within the reference area (1023) while moving the position. As a filter result of the reference area (1023), a Histogram of Gradients (HoG) representing the accumulated gradient magnitude for each directional prediction mode is generated. The 35th directional prediction mode having the largest gradient magnitude from the HoG can be determined as an intra prediction mode candidate.

[0188] In other embodiments, the size and type of a given filter may be set to various values.

[0189] In FIG. 10c, intra prediction mode candidates can be derived from one or more sub-blocks selected based on the area of ​​the sub-blocks or the shape of the sub-blocks within the reference region (1033).

[0190] In one embodiment, an intra prediction mode candidate may be derived from a subblock having a large area within the reference region (1033). Since the area of ​​the subblock predicted by intra prediction mode 50 within the reference region (1033) is the largest, intra prediction mode 50 may be determined to be the intra prediction mode candidate.

[0191] In one embodiment, an intra prediction mode candidate may be derived from at least one of the sub-blocks having a shape similar to that of the current block. Since the shape of the corresponding block (1031) is identical to that of the current block, and the shape of the sub-block predicted by the 50th intra prediction mode within the reference region (1033) is most similar to that of the corresponding block (1031), the 50th intra prediction mode may be determined to be the intra prediction mode candidate.

[0192] FIG. 11 is a flowchart of an image encoding method according to one embodiment of the present disclosure.

[0193] In step S1110, the video encoding device determines an intra prediction mode of the current block. The determined intra prediction mode may indicate an optimal prediction mode of the current block.

[0194] In step S1120, the video encoding device encodes the current block using the intra prediction mode of the current block. Specifically, the video encoding device performs intra prediction on the current block based on the intra prediction mode of the current block, thereby generating a prediction block of the current block. The video encoding device generates a residual block of the current block based on the difference between the original block of the current block and the prediction block or the difference between the original reconstructed block of the current block and the prediction block. The video encoding device encodes the residual block.

[0195] In another embodiment, step S1120 may be performed after step S1130 or after step S1140.

[0196] In step S1130, the image encoding device configures an MPM list based on the prediction mode of the surrounding blocks.

[0197] The video encoding device constructs an MPM list using the MPM list generation method described in FIG. 7. An MPM list according to the present disclosure may be generated when the prediction mode of a neighboring block is not an intra prediction mode or when a neighboring block cannot be used for prediction of the current block.

[0198] In step S1140, if the intra prediction mode of the current block is one of the MPM candidates, the video encoding device encodes information about the intra prediction mode of the current block using the MPM list.

[0199] Specifically, when the intra prediction mode of the current block is one of the MPM candidates, the video encoding device can encode syntax mpm_flag indicating whether the intra prediction mode of the current block is derived based on the prediction mode of the surrounding block, syntax not_planar_flag indicating that it is not a planar mode, and syntax mpm_idx for identifying an MPM candidate in the MPM list. When mpm_flag indicates that the MPM list is used and not_planar_flag indicates that the planar mode is not applied, mpm_idx can be encoded. When mpm_flag indicates that the MPM list is not used or not_planar_flag indicates that the planar mode is applied, mpm_idx can be unencoded.

[0200] On the other hand, if the intra prediction mode of the current block is not one of the MPM candidates, the video encoding device may encode a syntax mpm_remainder for identifying the intra prediction mode of the current block from the remaining prediction modes excluding the MPM candidates among the intra prediction modes.

[0201] The video encoding device can signal encoding information of the current block and encoding information of the intra prediction mode of the current block to the video decoding device.

[0202] FIG. 12 is a flowchart of an image decoding method according to one embodiment of the present disclosure.

[0203] In step S1210, the image decoding device obtains information indicating whether the intra prediction mode of the current block is derived based on the prediction mode of the surrounding blocks. In other words, the image decoding device decodes mpm_flag.

[0204] If mpm_flag indicates that the intra prediction mode of the current block is not derived based on the prediction modes of the surrounding blocks, the video decoding device decodes the syntax mpm_remainder for identifying the intra prediction mode of the current block. The video decoding device can determine the intra prediction mode of the current block using the value of mpm_remainder.

[0205] In step S1220, if mpm_flag indicates that the intra prediction mode of the current block is derived based on the prediction mode of the surrounding blocks, the image decoding device obtains information indicating whether the intra prediction mode of the current block is a planar mode. In other words, the image decoding device decodes not_planar_flag.

[0206] When not_planar_flag indicates that the planar mode is applied to the current block, the image decoding device determines that the prediction mode of the current block is the planar mode.

[0207] In step S1230, when not_planar_flag indicates that the planar mode is not applied to the current block, the image decoding device obtains intra prediction mode information of the current block. In other words, the image decoding device decodes mpm_idx.

[0208] In step S1240, the image decoding device configures an MPM list based on the prediction mode of the surrounding blocks.

[0209] If the prediction mode of the surrounding block is not an intra prediction mode or the surrounding block cannot be used for prediction of the current block, the image decoding device constructs an MPM list using the MPM list generation method described in FIG. 7.

[0210] In step S1250, the image decoding device determines the intra prediction mode of the current block from the MPM list using the intra prediction mode information of the current block.

[0211] Specifically, the video decoding device determines an MPM candidate matching mpm_idx among the MPM candidates in the MPM list as the intra prediction mode of the current block.

[0212] In step S1260, the image decoding device decodes the current block using the intra prediction mode of the current block.

[0213] Specifically, the video decoding device generates a prediction block of the current block by performing intra prediction on the current block based on the intra prediction mode of the current block. The video decoding device decodes the residual block of the current block from the bitstream and generates a reconstructed block of the current block from the residual block and the prediction block.

[0214] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0215] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0216] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0217] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

Claims

1. In the video decryption method, Step of determining the reference area of the current block; A step of deriving an intra prediction mode candidate of the current block based on information of the above reference area; A step of constructing a candidate list of intra prediction modes based on the intra prediction mode candidates of the current block; and A step of performing intra prediction for the current block based on the candidate list. A method for decrypting an image including:

2. In paragraph 1, The step of determining the reference area of the current block is: A step of determining that the reference area of the current block is an area indicated by a reference vector on a reference picture or a current picture, wherein the reference vector is used to predict a surrounding block of the current block. A method for decrypting an image including:

3. In paragraph 1, The step of determining the reference area of the current block is: A step of determining that the reference area of the current block is one of a region selected from candidate regions indicated by a plurality of reference vectors on the reference pictures or the current picture, an overlapping region, or the entire region. A method for decrypting an image including:

4. In paragraph 3, The above selected area is, An area selected from the candidate areas based on the weights of areas indicated by the plurality of reference vectors relative to one of the corresponding blocks in the reference pictures or the surrounding blocks in the current picture, A method for decoding an image in which corresponding blocks in the above reference pictures are areas having the same relative positions as the surrounding blocks in the above current picture.

5. In paragraph 3, The above selected area is, A method for decoding an image, wherein the region is selected from the candidate regions based on the number of sub-blocks in each candidate region.

6. In paragraph 1, A step of adjusting the reference area of the current block based on at least one of a predefined offset or a predefined scale ratio. A method for decrypting an image further comprising:

7. In paragraph 1, The step of deriving the intra prediction mode candidate is as follows: A step of determining the intra prediction mode candidate based on the intra prediction mode of at least one sub-block including a pixel at a preset position among the sub-blocks within the reference area of the current block. A method for decrypting an image including:

8. In paragraph 1, The step of deriving the intra prediction mode candidate is as follows: A step of determining the intra prediction mode candidate based on the areas and prediction modes of sub-blocks within the reference area of the current block. A method for decrypting an image including:

9. In paragraph 1, The step of deriving the intra prediction mode candidate is as follows: A step of determining the intra prediction mode candidate based on the shapes and prediction modes of sub-blocks within the reference area of the current block. A method for decrypting an image including:

10. In paragraph 1, The step of deriving the intra prediction mode candidate is as follows: A step of obtaining gradient information of the reference area by applying a predetermined filter to the reference area of the current block; and A step of determining the intra prediction mode candidate using the gradient information of the above reference area. A method for decrypting an image including:

11. In paragraph 1, The step of constructing a candidate list of the above intra prediction modes is: A step of constructing the candidate list based on the prediction mode of the surrounding block; and A step of adding the intra prediction mode candidate to the candidate list. A method for decrypting an image including:

12. In paragraph 1, The step of constructing a candidate list of the above intra prediction modes is: A step of replacing the prediction mode of the above-mentioned surrounding block with the intra prediction mode candidate; and A step of constructing the candidate list based on the replaced intra prediction mode of the above-mentioned surrounding block and the intra prediction mode of another surrounding block. A method for decrypting an image including:

13. In the video encoding method, Step of determining the reference area of the current block; A step of deriving an intra prediction mode candidate of the current block based on information of the above reference area; A step of constructing a candidate list of intra prediction modes based on the intra prediction mode candidates of the current block; and A step of performing intra prediction for the current block based on the candidate list. A method of encoding an image including:

14. A method for transmitting data including a bitstream for an image, A step of generating a bitstream for the above image; and A step of transmitting data including the above bitstream Including, The step of generating the above bitstream is: Step of determining the reference area of the current block; A step of deriving an intra prediction mode candidate of the current block based on information of the above reference area; A step of constructing a candidate list of intra prediction modes based on the intra prediction mode candidates of the current block; and A step of performing intra prediction for the current block using the above candidate list. How to include.

Images