Method and device for video coding using temporal MPM candidate

Abstract

The present embodiment discloses a video coding method and device using a temporal MPM candidate in intra prediction. In the present embodiment, an image decoding device configures a candidate list of a current block. The current block is included in an inter slice and is intra-predicted. The candidate list includes candidate intra modes. The image decoding device selects at least one reference picture in relation to the current block. The image decoding device obtains at least one intra mode from the at least one reference picture. The image decoding device updates the candidate list by adding the at least one intra mode to the candidate list.

Description

Method and device for video coding using temporal MPM candidates

[0001] The present disclosure relates to a video coding method and apparatus using temporal MPM candidates in intra prediction.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] The present disclosure aims to provide a video coding method and device for adding an intra mode obtained from a picture other than the current picture to the MPM (Most Probable Mode) list when constructing the MPM list in intra prediction.

[0007] According to an embodiment of the present disclosure, a method for restoring a current block, performed by a video decoding device, is provided, comprising: configuring a candidate list of the current block, wherein the current block is included in an inter-slice and is intra-predicted, and the candidate list includes candidate intra modes; selecting at least one reference picture in relation to the current block; obtaining at least one intra mode from the at least one reference picture; and updating the candidate list by adding the at least one intra mode to the candidate list.

[0008] According to another embodiment of the present disclosure, a method for encoding a current block, performed by a video encoding device, is provided, comprising: configuring a candidate list of the current block, wherein the current block is included in an inter-slice and is intra-predicted, and the candidate list includes candidate intra modes; selecting at least one reference picture in relation to the current block; obtaining at least one intra mode from the at least one reference picture; and updating the candidate list by adding the at least one intra mode to the candidate list.

[0009] According to another embodiment of the present disclosure, a method for providing video data to a video decoding device is provided, the method comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding the video data comprises: configuring a candidate list of a current block, wherein the current block is included in an inter-slice and is intra-predicted, and the candidate list includes candidate intra modes; selecting at least one reference picture in relation to the current block; obtaining at least one intra mode from the at least one reference picture; and updating the candidate list by adding the at least one intra mode to the candidate list.

[0010] As described above, according to the present embodiment, when constructing an MPM list in intra prediction, a video coding method and device are provided that add an intra mode obtained from a picture other than the current picture to the MPM list, thereby making it possible to improve video encoding efficiency and enhance video quality.

[0011] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0012] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0013] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0014] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0015] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0016] Figure 6 is an example diagram showing pixels used in the MPM (Most Probable Mode) configuration.

[0017] FIG. 7 is an exemplary diagram showing a temporal hierarchy in a hierarchical prediction structure according to one embodiment of the present disclosure.

[0018] FIG. 8 is an exemplary diagram showing the occurrence rate of predictions according to layers according to one embodiment of the present disclosure.

[0019] FIG. 9 is an exemplary diagram showing temporal hierarchies in a hierarchical prediction structure according to another embodiment of the present disclosure.

[0020] FIG. 10 is an exemplary diagram showing the occurrence rate of predictions according to layers according to another embodiment of the present disclosure.

[0021] Figure 11 is an example diagram showing a problem according to the configuration of the MPM list.

[0022] FIG. 12 is an exemplary diagram showing a solution to a problem according to the configuration of an MPM list according to one embodiment of the present disclosure.

[0023] FIGS. 13A to 13C are exemplary diagrams showing selection of reference pictures according to one embodiment of the present disclosure.

[0024] FIG. 14 is an exemplary diagram showing a preset position in a reference picture according to one embodiment of the present disclosure.

[0025] FIG. 15a and FIG. 15b are exemplary diagrams showing acquisition of intra mode according to one embodiment of the present disclosure.

[0026] FIGS. 16A to 16C are exemplary diagrams showing acquisition of intra mode according to another embodiment of the present disclosure.

[0027] FIG. 17 is a flowchart illustrating a method of encoding a current block performed by an image encoding device according to one embodiment of the present disclosure.

[0028] FIG. 18 is a flowchart illustrating a method for restoring a current block performed by an image decoding device according to one embodiment of the present disclosure.

[0029] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0030] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0031] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0032] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0033] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0034] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0035] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0036] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0037] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0038] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0039] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0040] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0041] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0042] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0043] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0044] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0045] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0046] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0047] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0048] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0049] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0050] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0051] Various methods can be used to minimize the number of bits required to encode motion information.

[0052] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0053] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0054] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0055] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0056] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0057] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0058] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0059] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0060] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0061] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0062] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0063] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0064] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0065] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0066] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0067] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0068] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0069] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0070] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.

[0071] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0072] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0073] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0074] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0075] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0076] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0077] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0078] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0079] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0080] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0081] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0082] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0083] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0084] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0085] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0086] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0087] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0088] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0089] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0090] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0091] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0092] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0093] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0094] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0095] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0096] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0097] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0098] The present embodiment relates to encoding and decoding of images (videos) as described above. More specifically, a video coding method and device are provided for adding an intra mode obtained from a picture other than the current picture to the MPM (Most Probable Mode) list when constructing the MPM list in intra prediction.

[0099] The following embodiments may be performed by a prediction unit (120) within a video encoding apparatus. Additionally, the following embodiments may be performed by a prediction unit (540) within a video decoding apparatus.

[0100] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0101] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0102] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0103] The decoder side includes all or part of the inverse quantization unit (160), the inverse transform unit (165), the prediction unit (120), the adder (170), the loop filter unit (180), and the memory (190) in the video encoding device illustrated in FIG. 1. Alternatively, the decoder side includes all or part of the inverse quantization unit (520), the inverse transform unit (530), the prediction unit (540), the adder (550), the loop filter unit (560), and the memory (570) in the video decoding device illustrated in FIG. 5. With respect to a series of decoding processes, the decoder side of the video encoding device and the decoder side of the video decoding device perform the same operation. The video encoding device determines information related to the operation of the decoder side and signals the determined information to the video decoding device. The video decoding device can decode the signaled information and operate the decoder side based on the decoded information.

[0104] I-1. Intra prediction

[0105] Intra prediction uses pixel information within the same picture to predict pixel values ​​of the current block to be encoded. Intra prediction can select the most appropriate mode among multiple intra prediction modes based on the characteristics of the image, and then use the selected mode to predict the current block. An image encoding device selects one of the multiple intra prediction modes and encodes the current block using the selected mode. The image encoding device can then transmit information about the selected mode to an image decoding device.

[0106] HEVC technology uses a total of 35 intra prediction modes, including 33 angular modes and 2 non-angular modes for intra prediction. However, as the spatial resolution of an image increases from 720×480 to 2048×1024 or 8192×4096, the size of the prediction block unit also increases, and accordingly, the need to add more diverse intra prediction modes has increased. As illustrated in Fig. 3a, VVC technology can utilize more diverse prediction directions compared to previous technologies by using 65 more detailed prediction modes for intra prediction. With the introduction of WAIP (Wide Angular Intra Prediction), intra modes -14 to -1 and 67 to 80, which are directional modes with larger angles, can be used for rectangular blocks depending on the aspect ratio of the block.

[0107] According to VVC technology, intra prediction modes can represent 67 modes that can be signaled according to the aspect ratio of a block, from -14 to 80 intra prediction modes, including non-directional prediction modes such as Planar and DC modes. In addition to intra prediction techniques that use intra prediction modes, IBC (Intra Block Copy), palette, MIP (Matrix-based Intra Prediction), BDPCM (Block-based Delta Pulse Code Modulation) technology, and inter prediction technology can be utilized for prediction of the current block.

[0108] I-2. MPM (Most Probable Mode) Technology

[0109] As described above, in intra prediction, a predictor for the luma channel can be generated based on 67 Intra Prediction Modes (IPMs). The 67 IPMs refer to 67 intra prediction modes that can be signaled according to the aspect ratio of a block among prediction modes -14 to 80, including non-directional prediction modes such as Planar and DC modes. When generating a predictor using one of the 67 prediction modes, the video encoding device signals the prediction mode using the Most Probable Mode (MPM) to efficiently transmit prediction mode information.

[0110] The MPM technique utilizes the property that, when encoding blocks using an intra prediction mode, the prediction modes of neighboring blocks are likely to be similar. The MPM technique selects six MPM candidates based on the intra modes used by the neighboring blocks of the current block, the adjacent modes of the used modes, and the statistically frequently used modes. The set of six selected MPM candidates is called an MPM list. If the intra prediction mode of the current block is included in the MPM list, the MPM index corresponding to the intra prediction mode of the current block among the candidates included in the list is encoded. On the other hand, if the intra prediction mode of the current block is not included in the six MPM candidates, the six MPM candidates are excluded from the 67 intra modes, and an MPM remainder, which is a set of intra modes, is constructed. Thereafter, the intra prediction mode of the current block can be encoded using the MPM remainder.

[0111] As in the example of Fig. 6, with respect to blocks containing pixel A located to the left of the lower left pixel of the current block and pixel B located above the upper right pixel, the prediction modes of each block are defined as modeA and modeB. Based on modeA and modeB, six MPM candidates can be selected as follows to generate an MPM list. If the current block is located at the boundary of a CTU, tile, slice, sub-picture, picture, etc., and pixel A or pixel B is not available, the prediction mode of the block containing the corresponding pixel is regarded as Planar. Thereafter, the MPM list can be generated according to the following MPM list generation algorithm (hereinafter, 'generation algorithm').

[0112] First, if modeA and modeB are the same and modeA is greater than INTRA_DC, {Planar, modeA, 2 + ((modeA + 61) % 64), 2 + ((modeA - 1 ) % 64), 2 + ((modeA + 60) % 64), 2 + (modeA % 64)} are selected as MPM candidates.

[0113] Next, if modeA and modeB are not equal and either modeA or modeB is greater than INTRA_DC, MPM candidates are constructed as follows. At this time, minAB = Min(modeA, modeB) and maxAB = Max(modeA, modeB) are defined.

[0114] If both modeA and modeB are greater than INTRA_DC and maxAB - minAB = 1, {Planar, modeA, modeB, 2 + ((minAB + 61) % 64), 2 + ((maxAB - 1) % 64), 2 + ((minAB + 60) % 64)} are selected as MPM candidates.

[0115] If both modeA and modeB are greater than INTRA_DC and maxAB - minAB ≥ 62, {Planar, modeA, modeB, 2 + ((minAB - 1) % 64), 2 + ((maxAB + 61 ) % 64), 2 + (minAB % 64)} are selected as MPM candidates.

[0116] If both modeA and modeB are greater than INTRA_DC and maxAB - minAB = 2, {Planar, modeA, modeB, 2 + ((minAB - 1) % 64), 2 + ((minAB + 61) % 64), 2 + ((maxAB - 1) % 64)} are selected as MPM candidates.

[0117] If both modeA and modeB are greater than INTRA_DC and 2 < maxAB - minAB < 62, {Planar, modeA, modeB, 2 + ((minAB + 61) % 64), 2 + ((minAB - 1) % 64), 2 + ((maxAB + 61) % 64)} are selected as MPM candidates.

[0118] If modeA and modeB are not equal, and either modeA or modeB is greater than INTRA_DC, {Planar, maxAB, 2 + ((maxAB + 61) % 64), 2 + ((maxAB - 1) % 64), 2 + ((maxAB + 60) % 64), 2 + (maxAB % 64)} are selected as MPM candidates.

[0119] Additionally, if both modeA and modeB are equal to or less than INTRA_DC, {Planar, INTRA_DC, INTRA_ANGULAR50, INTRA_ANGULAR18, INTRA_ANGULAR46, INTRA_ANGULAR54} are selected as MPM candidates.

[0120] Meanwhile, when using MPM, the video decoding device parses the intra prediction mode of the current block as shown in Table 1.

[0121]

[0122] First, if intra_luma_ref_idx, which is a reference line index indicating one of multiple reference lines, is 0, intra_luma_mpm_flag, which is a flag indicating whether to use MPM, can be signaled from the video encoding device to the video decoding device. If intra_luma_mpm_flag is true and intra_luma_ref_idx is 0, intra_luma_not_planar_flag, which is a flag indicating whether to use Planar mode, can be signaled from the video encoding device to the video decoding device. If intra_luma_not_planar_flag is false, the intra prediction mode is set to Planar mode, and if intra_luma_not_planar_flag is true, intra_luma_mpm_idx can be additionally signaled. If intra_luma_not_planar_flag does not exist, it can be inferred to be 1.

[0123] Meanwhile, if intra_luma_ref_idx is not 0, planar mode is not used. Therefore, intra_luma_not_planar_flag is not transmitted and is considered true. In addition, since intra_luma_not_planar_flag is true, intra_luma_mpm_idx can be additionally signaled.

[0124] Next, if intra_luma_mpm_flag is false, MPM reminder is signaled as intra prediction mode.

[0125] II. Embodiments according to the present disclosure

[0126] Encoding of an intra prediction mode according to the present disclosure can be particularly efficient in encoding and decoding intra prediction blocks existing in an inter-slice. The occurrence ratios of blocks encoded with intra prediction, inter prediction, and IBC in an inter-slice can be expressed as in Table 2. Table 2 shows the experimental results based on RA configuration (Random Access Configuration) and QP (Quantization Parameter) 22, 27, 32, and 37 in the software of VVC test model version 22.0.

[0127]

[0128] The frame rate and intra period (the period in which intra slices are used) of the video are as shown in Table 2, and the GOP (Group of Pictures) value of all videos is the same at 32. According to Table 2, the proportion of blocks encoded according to intra prediction in an inter-slice is approximately 1% to 50%, and in some videos, it exceeds 50%. Conventional methods related to intra-mode encoding mainly use intra-prediction blocks within an intra-slice, so when constructing the MPM list of the current block, the intra-modes of the surrounding blocks within the current picture are added to the MPM list. However, as shown in the experimental results in Table 2, many blocks within an inter-slice are encoded based on intra-prediction. For a block intra-predicted within an inter-slice, there may not be any blocks encoded according to intra-prediction around it. If an MPM list is constructed using surrounding blocks within the current picture encoded according to intra-prediction, the encoding efficiency related to the intra-prediction mode of the current block may be significantly reduced. Therefore, using the technology according to the present disclosure, intra prediction can be encoded more efficiently in inter-slices.

[0129] FIG. 7 is an exemplary diagram showing a temporal hierarchy in a hierarchical prediction structure according to one embodiment of the present disclosure.

[0130] In the example of Fig. 7, the GOP is 32 and the intra period is 64. L0 represents an intra slice, and all blocks within an intra slice are encoded and decoded according to intra prediction. L1 to L6 represent inter slices, and blocks within an inter slice are encoded and decoded according to inter prediction or intra prediction. An inter slice can use a previously decoded picture as a reference picture. A picture of an L1 layer can use a picture of an L0 layer as a reference picture, and a picture of an L2 layer can use a picture of an L0 layer and a picture of an L1 layer as reference pictures. When encoding according to inter prediction in an inter slice, a picture existing in a layer below the current picture can be utilized as a reference picture.

[0131] Regarding images with 64 intra-period values ​​among the images according to Table 2, the occurrence ratio of predictions by layer is as shown in Figure 8. The lower the layer, the higher the usage ratio of intra prediction. In the L1 layer of a specific image, more than 50% of the blocks are encoded according to intra prediction. The example of Figure 8 shows that it is important to improve the encoding efficiency of intra prediction in inter-slice.

[0132] FIG. 9 is an exemplary diagram showing temporal hierarchies in a hierarchical prediction structure according to another embodiment of the present disclosure.

[0133] In the example of Fig. 9, the GOP is 32 and the intra period is 32. L0 represents an intra slice, and L1 to L5 represent inter slices.

[0134] Regarding images with 32 intra-period values ​​among the images according to Table 2, the occurrence ratio of predictions by layer is as shown in Figure 10. The lower the layer, the higher the usage ratio of intra prediction. In the L1 layer of a specific image, more than 50% of the blocks are encoded according to intra prediction. The example in Figure 10 shows that it is important to improve the encoding efficiency of intra prediction in inter-slice.

[0135] In the process of generating an MPM list according to the existing technology, since the positions of the anchor pixels used to determine the anchor block are fixed to L and A, as shown in Fig. 6, the intra modes of blocks other than the blocks containing pixels L and A cannot be used in the process of generating the MPM list. When constructing an MPM list in an inter-slice, there may be many blocks encoded with Inter and IBC around the current block. If the MPM list is constructed according to the existing method, intra prediction may not proceed well. Therefore, it may be more optimal to obtain an intra mode from a picture other than the current picture and add the obtained intra mode to the MPM list.

[0136] As shown in Fig. 11, when each block containing pixels L and A is determined as a reference block to generate an MPM list, the intra prediction modes of the reference block are all replaced with Planar. Since the two modes of the reference block are the same non-directional mode, the MPM list is generated as {0 (Planar), 1 (DC), 50, 18, 46, 54} according to the generation algorithm described above. When the intra prediction mode of the current block is 19, mode 19, which does not exist in the MPM list (based on the MPM reminder), is selected, so the video encoding device must consume many bits to transmit the mode.

[0137] According to the present disclosure, when constructing an MPM list, as shown in FIG. 12, by adding the intra mode of a reference picture to the MPM list, an MPM list can be generated using mode 19. Since mode 19 selected by the current block exists in the MPM list, bits for transmitting mode 19 can be reduced, and encoding efficiency can be increased.

[0138] The present disclosure focuses on the problem of reduced encoding efficiency of intra prediction when generating an MPM list for a current block, as in the example described above, due to the presence of many blocks encoded using inter-prediction or IBC around the current block. Since the aforementioned problem occurs more frequently in inter-slices, the effects of the present disclosure may be more pronounced in inter-slices.

[0139] The following embodiments are described with a focus on a video decoding device, but can be implemented in the same or similar manner in a video encoding device. Alternatively, the following embodiments are described with a focus on the decoder side of a video decoding device, but can also be implemented in the same or similar manner in the decoder side of a video encoding device.

[0140] In the present disclosure, when there is an available reference picture during the process of generating an MPM list, the video decoding device can select a reference picture, obtain an intra mode from the selected picture, and add it to the MPM list. Hereinafter, the intra mode present in the reference picture is referred to as a temporal MPM candidate.

[0141] As an example, an MPM list with an intra mode obtained from a reference picture added may be used. In the present embodiment, the video decoding device can configure an MPM list by adding the intra mode obtained from the reference picture to the MPM list, and decode the intra prediction mode of the current block using the configured MPM list. The present embodiment can be implemented in three steps. In the first step, the video decoding device selects a reference picture. In the second step, the video decoding device obtains an intra mode from the selected reference picture and adds the obtained intra mode to the MPM list. In the third step, the video decoding device decodes the intra prediction mode of the current block based on the MPM list.

[0142] In the first step, the video decoding device selects a reference picture from among the reference pictures in the reference picture list. The video decoding device may select one or more reference pictures.

[0143] As an example, a video decoding device can select a preset reference picture. The video decoding device can select a preset reference picture, or use a reference picture indicated at a high level, such as a picture header or a slice header. When selecting a preset reference picture, a co-located picture, a specific picture in a reference picture list, etc. can be utilized as the preset reference picture. The co-located picture can be a reference picture set at a high level, such as a picture header or a slice header, for generating a motion vector predictor. The specific picture in the reference picture list can be a specific picture (e.g., the first picture) in reference picture list 0 (L0) or a specific picture (e.g., the second picture) in reference picture list 1 (L1). Which reference picture list to use and which picture within the reference picture list to use can be defined in advance according to an agreement between the video encoding device and the video decoding device.

[0144] FIGS. 13A to 13C are exemplary diagrams showing selection of reference pictures according to one embodiment of the present disclosure.

[0145] As another example, a video decoding device may select a reference picture for an inter-predicted block surrounding the current block. A neighboring block of the current block may refer to a block within a certain distance from the current block within the same picture as the current block. In this case, the neighboring block may include a reference block within the same picture.

[0146] When the reference picture is singular, the video decoding device can select the corresponding reference picture, as in the example of Fig. 13a. When the reference picture of a block encoded according to inter prediction around the current block is singular, the video decoding device can select the corresponding reference picture. In Fig. 13a, the reference picture index RefIdx indicates a reference picture within the reference picture list (L0 or L1).

[0147] When there are multiple reference pictures, the video decoding device can select a single reference picture that satisfies a predetermined condition, as in the example of FIG. 13b. For example, the video decoding device can select a single reference picture so as to satisfy the condition that the weight is the largest. When there are multiple reference pictures, the weight represents a value used for the weighted sum of reference blocks during bi-prediction. In FIG. 13b, since w0=1 and w1=7, the video decoding device selects reference picture 1 including the reference block with the largest weight. Including the example of FIG. 13b, the video decoding device can utilize, as predetermined conditions, a picture including a reference block with the largest weight, a picture having the smallest difference in POC (Picture Order Count) from the current picture, a picture existing in L0, a picture existing in L1, etc.

[0148] When there are multiple reference pictures, as in the example of FIG. 13c, the video decoding device can select multiple reference pictures that meet a predetermined condition. For example, when there is a block encoded according to inter prediction around the current block and there are two reference pictures related to the block, the video decoding device can select both reference pictures. Including the example of FIG. 13c, the video decoding device can utilize, as predetermined conditions, all reference pictures used for both predictions, pictures including reference blocks with a large weight (e.g., greater than a preset threshold), pictures with a POC difference from the current picture smaller (e.g., greater than a preset threshold), etc.

[0149] In the second step, the video decoding device can obtain an intra mode from a reference picture and add the obtained intra mode to the MPM list. The video decoding device can obtain one or more intra modes from each reference picture and add the obtained intra modes to the MPM list.

[0150] As an example, the image decoding device can set a pixel at a preset position in a selected reference picture as an anchor pixel, obtain an intra mode from the preset anchor pixel, and add the obtained intra mode to the MPM list. The preset position can be, as shown in FIG. 14, within a block at the same position as the current block, an adjacent position, a non-adjacent position, etc. The image decoding device can set pixels at various positions as anchor pixels, obtain an intra mode based on pixels at preset positions in the selected reference picture, and add the obtained intra mode to the MPM list.

[0151] As another example, the video decoding device may set a reference pixel by utilizing motion information around the current block in the selected reference picture, and add the intra mode of the set reference pixel to the MPM list. The video decoding device may set a pixel in a block at the same position as the current block in the reference picture by a block vector (BV) or a motion vector (MV) as a reference pixel by utilizing motion information around the current block, obtain an intra mode from the set reference pixel, and add the obtained intra mode to the MPM list.

[0152] FIG. 15a and FIG. 15b are exemplary diagrams showing acquisition of intra mode according to one embodiment of the present disclosure.

[0153] If there is a block decoded according to IBC or inter prediction around the current block, the image decoding device can obtain an intra mode from a reference picture by utilizing the block vector or motion vector of the block, and add the obtained intra mode to the MPM list.

[0154] In Fig. 15a, the video decoding device utilizes the block vector surrounding the current block. If the reference block surrounding the current block is decoded according to IBC, the video decoding device can obtain the intra mode of a block that is located at the same location as the current block (co-located block) within the reference picture by the block vector of the reference block, and add the obtained intra mode to the MPM list. In Fig. 15a, the 52nd intra mode, which is a temporal MPM candidate, can be added to the MPM list.

[0155] In Fig. 15b, the video decoding device utilizes the motion vectors surrounding the current block. If the reference block surrounding the current block is decoded according to inter prediction, the video decoding device can obtain the intra mode of a block located at the same position as the current block within the reference picture by the motion vector of the reference block, and add the obtained intra mode to the MPM list. In Fig. 15b, the 52nd intra mode, which is a temporal MPM candidate, can be added to the MPM list.

[0156] FIGS. 16A to 16C are exemplary diagrams showing acquisition of intra mode according to another embodiment of the present disclosure.

[0157] The video decoding device can obtain an intra mode from a reference picture by utilizing block vectors or motion vectors recursively generated from used block vectors or motion vectors surrounding the current block, and add the obtained intra mode to the MPM list. The above-described process can proceed as follows.

[0158] The video decoding device searches for block vectors and motion vectors around the current block, and obtains a reference block indicated by the searched block vector or motion vector (S10).

[0159] The video decoding device searches for a block vector and a motion vector within the acquired reference block, and obtains a reference block indicated by the searched block vector or motion vector (S20 to S26).

[0160] A video decoding device searches for an intra-predicted block by repeating recursive steps N times (where N is a natural number greater than or equal to 2). The video decoding device regards the found intra-predicted block as a final reference block and generates a new block vector or motion vector indicating the relationship between the current block and the final reference block (S30). The video decoding device may repeat the recursive steps N times until the intra-predicted block is found. The new block vector or motion vector may be generated based on N pieces of recursively searched motion information. If the intra-predicted block is not found even after repeating the recursive steps N times, the video decoding device may omit adding the intra mode to the MPM list.

[0161] The video decoding device obtains an intra mode of a block that is located at the same position as the current block within a reference picture by a new block vector or a motion vector, and adds the obtained intra mode to the MPM list (S40).

[0162] In Fig. 16a, the video decoding device utilizes a block vector recursively generated based on a block vector. The video decoding device can recursively generate a block vector from blocks decoded according to IBC around the current block, obtain an intra mode by utilizing the block vector recursively generated from a selected reference picture, and add the obtained intra mode to the MPM list. In Fig. 16a, the 19th intra mode, which is a temporal MPM candidate, can be added to the MPM list. As described above, the reference picture can be selected according to the first step.

[0163] In Fig. 16b, the video decoding device utilizes a motion vector generated recursively based on the motion vector. The video decoding device can recursively generate a motion vector from a block decoded according to inter prediction around the current block, obtain an intra mode by utilizing the recursively generated motion vector from the selected reference picture, and add the obtained intra mode to the MPM list. In Fig. 16b, the 54th intra mode, which is a temporal MPM candidate, can be added to the MPM list. As described above, the reference picture can be selected according to the first step.

[0164] In Fig. 16c, the video decoding device utilizes a motion vector recursively generated based on a block vector and a motion vector. The video decoding device obtains a motion vector from a block decoded according to inter prediction around the current block, and obtains a block vector existing in a reference block indicated by the obtained motion vector or a reference block indicated by the motion vector. The video decoding device may obtain a motion vector by repeating the process, for example, four times, and may obtain an intra mode by utilizing the motion vector obtained from the selected reference picture, and add the obtained intra mode to the MPM list. In Fig. 16c, the 54th intra mode, which is a temporal MPM candidate, may be added to the MPM list. As described above, the reference picture may be selected according to the first step.

[0165] In the third step, the intra prediction mode can be encoded or decoded using the MPM list constructed by adding temporal MPM candidates.

[0166] As an example, a video encoding device can encode an intra prediction mode of a current block using an MPM list constructed by adding temporal MPM candidates. If the intra prediction mode is included in the MPM list, the video encoding device can encode an index indicating the intra prediction mode within the MPM list. On the other hand, if the intra prediction mode is not included in the MPM list, the video encoding device can encode the intra prediction mode based on an MPM reminder.

[0167] As another example, a video decoding device can decode the intra prediction mode of the current block using an MPM list constructed by adding temporal MPM candidates. If an index indicating an intra prediction mode within the MPM list is parsed, the video decoding device can decode the intra prediction mode based on the parsed index and the MPM list. If the index is not parsed, the video decoding device can decode the intra prediction mode based on the MPM reminder.

[0168] Meanwhile, although an embodiment of adding a temporal MPM candidate in an inter-slice has been described, the method of adding an MPM candidate according to the present disclosure can also be utilized in an intra-slice. For example, in the first step, the current picture is used instead of a reference picture. In the second step, instead of selecting an intra mode from the reference picture, an intra mode can be obtained from a reference block in the current picture, and the obtained intra mode can be added to the MPM list. In the third step, the intra prediction mode of the current block can be encoded and decoded based on the constructed MPM list.

[0169] Hereinafter, using the examples of FIGS. 17 and 18, a method of constructing an MPM list by adding temporal MPM candidates in an inter-slice is described.

[0170] FIG. 17 is a flowchart illustrating a method of encoding a current block performed by an image encoding device according to one embodiment of the present disclosure.

[0171] The video encoding device constructs a candidate list of the current block (S1700).

[0172] The current block is included in the inter-slice and is intra-predicted. The candidate list is an MPM list and may include candidate intra modes selected according to an existing algorithm for generating MPM lists.

[0173] The video encoding device selects at least one reference picture in relation to the current block (S1702).

[0174] As an example, the video encoding device can obtain at least one reference picture, a preset reference picture or a reference picture indicated at a higher level.

[0175] As another example, the video encoding device can search for an inter-predicted block among blocks existing within a certain distance from a current block, and obtain at least one reference picture of the searched block as at least one reference picture.

[0176] As another example, when the searched block has multiple reference pictures, the image encoding device can obtain a reference picture having the largest weight among the multiple reference pictures.

[0177] The video encoding device obtains at least one intra mode from at least one reference picture (S1704).

[0178] As an example, an image encoding device may set anchor pixels as pixels at preset locations in a selected reference picture. The image encoding device may obtain at least one intra mode based on the set anchor pixels.

[0179] As another example, an image encoding device can obtain motion information of a block located within a certain distance from a current block. A motion vector or a block vector can be used as the motion information. The image encoding device can set reference pixels by applying the motion information to a block co-located with the current block within a selected reference picture. The image encoding device can obtain at least one intra mode based on the set reference pixels.

[0180] As another example, a video encoding device can obtain first motion information of a block existing within a certain distance from a current block. Until the Nth (wherein N is a natural number greater than or equal to 2) motion information indicates an intra-predicted block, the video encoding device can recursively generate second motion information to Nth motion information based on the first motion information. A motion vector or a block vector can be used as each motion information. When the Nth motion information indicates an intra-predicted block, the video encoding device can generate new motion information based on the first motion information to Nth motion information. The video encoding device can set reference pixels by applying the new motion information to a block co-located with the current block within a selected reference picture. The video encoding device can obtain at least one intra mode based on the set reference pixels. On the other hand, when the Nth motion information does not indicate an intra-predicted block, the video encoding device can omit the process of obtaining at least one intra mode.

[0181] The video encoding device updates the candidate list by adding at least one intra mode to the candidate list (S1706).

[0182] At least one additional intra mode represents a temporal candidate.

[0183] The video encoding device determines the intra prediction mode of the current block (S1708). For example, in terms of rate distortion optimization, the video encoding device can determine the intra prediction mode of the current block.

[0184] The video encoding device determines a candidate index corresponding to the intra prediction mode of the current block based on the updated candidate list (S1710). The candidate index may represent an MPM index.

[0185] The video encoding device encodes the candidate index (S1712).

[0186] Thereafter, the video encoding device can generate a prediction block of the current block based on the intra prediction mode, and generate a residual block by subtracting the prediction block from the current block. The video encoding device can apply transformation / quantization to the residual block to generate quantized transform coefficients, and encode the quantized transform coefficients to generate a bitstream.

[0187] FIG. 18 is a flowchart illustrating a method for restoring a current block performed by an image decoding device according to one embodiment of the present disclosure.

[0188] The video decoding device constructs a candidate list of the current block (S1800).

[0189] The current block is included in the inter-slice and is intra-predicted. The candidate list is an MPM list and may include candidate intra modes selected according to an existing algorithm for generating MPM lists.

[0190] The video decoding device selects at least one reference picture in relation to the current block (S1802).

[0191] As an example, the video decoding device can obtain at least one reference picture, a preset reference picture or a reference picture indicated at a higher level.

[0192] As another example, the video decoding device can search for an inter-predicted block among blocks existing within a certain distance from a current block, and obtain at least one reference picture of the searched block as at least one reference picture.

[0193] As another example, when the searched block has multiple reference pictures, the image decoding device can obtain a reference picture having the largest weight among the multiple reference pictures.

[0194] The video decoding device obtains at least one intra mode from at least one reference picture (S1804).

[0195] As an example, an image decoding device may set anchor pixels as pixels at preset locations in a selected reference picture. The image decoding device may obtain at least one intra mode based on the set anchor pixels.

[0196] As another example, an image decoding device can obtain motion information of a block located within a certain distance from a current block. A motion vector or a block vector can be used as the motion information. The image decoding device can set reference pixels by applying the motion information to a block co-located with the current block within a selected reference picture. The image decoding device can obtain at least one intra mode based on the set reference pixels.

[0197] As another example, an image decoding device may obtain first motion information of a block existing within a certain distance from a current block. Until the Nth (wherein N is a natural number greater than or equal to 2) motion information indicates an intra-predicted block, the image decoding device may recursively generate second motion information to Nth motion information based on the first motion information. A motion vector or a block vector may be used as each motion information. If the Nth motion information indicates an intra-predicted block, the image decoding device may generate new motion information based on the first motion information to Nth motion information. The image decoding device may apply the new motion information to a block co-located with the current block within a selected reference picture to set reference pixels. The image decoding device may obtain at least one intra mode based on the set reference pixels. On the other hand, if the Nth motion information does not indicate an intra-predicted block, the image decoding device may omit the process of obtaining at least one intra mode.

[0198] The video decoding device updates the candidate list by adding at least one intra mode to the candidate list (S1806).

[0199] At least one additional intra mode represents a temporal candidate.

[0200] The video decoding device decodes a candidate index from the bitstream (S1808). The candidate index may represent an MPM index.

[0201] The video decoding device extracts the intra prediction mode of the current block from the updated candidate list based on the candidate index (S1810).

[0202] Thereafter, the video decoding device generates a prediction block of the current block based on the intra prediction mode. The video decoding device can decode quantization transform coefficients from the bitstream and apply inverse quantization / inverse transformation to the quantization transform coefficients to generate a residual block. The video decoding device can generate a reconstructed block of the current block by adding the residual block and the prediction block.

[0203] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0204] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0205] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0206] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but rather to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0207]

[0208]

[0209] CROSS-REFERENCE TO RELATED APPLICATION

[0210] This patent application claims priority to Korean patent application No. 10-2024-0050288, filed in Korea on April 15, 2024, and Korean patent application No. 10-2025-0044808, filed in Korea on April 7, 2025, the entire contents of which are incorporated herein by reference.

Claims

1. In a method for restoring a current block performed by an image decryption device, A step of constructing a candidate list of the current block, wherein the current block is included in an inter slice and is intra predicted, and the candidate list includes candidate intra modes; A step of selecting at least one reference picture in relation to the current block; A step of obtaining at least one intra mode from at least one reference picture; and A step of updating the candidate list by adding at least one intra mode to the candidate list. A method comprising:

2. In paragraph 1, A step of decoding a candidate index from a bitstream; A step of extracting the intra prediction mode of the current block from the updated candidate list based on the candidate index; and A step of generating a prediction block of the current block based on the intra prediction mode. A method that includes more rules.

3. In paragraph 1, The step of selecting at least one reference picture above comprises: A method for obtaining at least one reference picture, a preset reference picture or a reference picture indicated at a higher level.

4. In paragraph 1, The step of selecting at least one reference picture above comprises: A step of searching for an inter-predicted block among blocks existing within a certain distance from the current block; and A step of obtaining at least one reference picture of the searched block as at least one reference picture. A method comprising:

5. In paragraph 4, The step of selecting at least one reference picture above comprises: A method for obtaining a reference picture having the largest weight among the plurality of reference pictures when the above-mentioned searched block has multiple reference pictures.

6. In paragraph 1, The step of obtaining at least one intra mode comprises: A step of setting anchor pixels as pixels at preset locations in a selected reference picture; and A step of obtaining at least one intra mode based on the set reference pixels A method comprising:

7. In paragraph 1, The step of obtaining at least one intra mode comprises: A step of obtaining movement information of a block existing within a certain distance from the current block; A step of setting reference pixels by applying the motion information to a block co-located with the current block within the selected reference picture; and A step of obtaining at least one intra mode based on the set reference pixels A method comprising:

8. In paragraph 7, The step of obtaining the above movement information is: A method for obtaining a motion vector or a block vector as the above motion information.

9. In paragraph 1, The step of obtaining at least one intra mode comprises: A step of obtaining first motion information of a block existing within a certain distance from the current block; and A step of recursively generating second motion information to the Nth motion information based on the first motion information until the Nth (where N is a natural number greater than or equal to 2) motion information indicates an intra-predicted block. A method comprising:

10. In paragraph 9, The step of obtaining at least one intra mode comprises: If the above Nth motion information indicates an intra-predicted block, A step of setting reference pixels by applying the first motion information to the Nth motion information to a block co-located with the current block within the selected reference picture; and A step of obtaining at least one intra mode based on the set reference pixels A method further comprising:

11. In a method for encoding a current block performed by a video encoding device, A step of constructing a candidate list of the current block, wherein the current block is included in an inter slice and is intra predicted, and the candidate list includes candidate intra modes; A step of selecting at least one reference picture in relation to the current block; A step of obtaining at least one intra mode from at least one reference picture; and A step of updating the candidate list by adding at least one intra mode to the candidate list. A method comprising:

12. In paragraph 11, A step of determining an intra prediction mode of the current block; A step of determining a candidate index corresponding to the intra prediction mode of the current block based on the updated candidate list; A step of generating a prediction block of the current block based on the intra prediction mode; and A step of encoding the above candidate index A method that includes more rules.

13. In paragraph 11, The step of obtaining at least one intra mode comprises: A step of obtaining movement information of a block existing within a certain distance from the current block; A step of setting reference pixels by applying the motion information to a block co-located with the current block within the selected reference picture; and A step of obtaining at least one intra mode based on the set reference pixels A method comprising:

14. In paragraph 13, The step of obtaining the above movement information is: A method for obtaining a motion vector or a block vector as the above motion information.

15. In a method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: A step of constructing a candidate list of a current block, wherein the current block is included in an inter slice and is intra predicted, and the candidate list includes candidate intra modes; A step of selecting at least one reference picture in relation to the current block; A step of obtaining at least one intra mode from at least one reference picture; and A step of updating the candidate list by adding at least one intra mode to the candidate list. A method comprising:

Images