Method for generating intra-prediction block for inter-prediction

Abstract

The present embodiment provides a method for generating an intra-prediction block for inter-prediction. In the present embodiment, an image decoding device configures a candidate list of a current block. Here, the candidate list includes intra-prediction mode candidates and is used for intra-prediction of the current block in an inter-prediction technology that utilizes intra-prediction. The image decoding device derives a preset number of intra-prediction mode candidates from the intra-prediction mode candidates in the candidate list to obtain a preset number of intra-prediction modes of the current block. The image decoding device generates a preset number of intra-prediction blocks on the basis of the preset number of intra-prediction modes. The image decoding device generates a final intra-prediction block of the current block by performing weighted fusion of the preset number of intra-prediction blocks.

Description

Method for generating an intra prediction block for inter prediction

[0001] The present disclosure relates to a video encoding / decoding method, an apparatus, and a recording medium for storing a bitstream, and more specifically, to a method for generating an intra-prediction block in an inter-prediction utilizing intra-prediction information.

[0002] The following description merely provides background information related to the present invention and does not constitute prior art.

[0003] Because video data contains a large amount of data compared to audio or still image data, storing or transmitting it as is without compression processing requires significant hardware resources, including memory.

[0004] Therefore, typically when storing or transmitting video data, the encoder compresses the video data for storage or transmission, and the decoder receives the compressed video data, decompresses it, and plays it. Such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves coding efficiency by more than 30% compared to HEVC.

[0005] However, as video size, resolution, and frame rates are gradually increasing, and the amount of data that needs to be encoded is also growing accordingly, a new compression technology is required that offers better encoding efficiency and higher image quality improvement effects than existing compression technologies.

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus for generating an intra prediction block in an inter prediction utilizing intra prediction information, and a recording medium for storing a bitstream generated by said image encoding method / apparatus.

[0007] According to an embodiment of the present disclosure, a method for restoring a current block performed by an image decoding device comprises the steps of: configuring a candidate list of the current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; deriving a predetermined number of intra prediction mode candidates from the intra prediction mode candidates in the candidate list to obtain a predetermined number of intra prediction modes of the current block; generating a predetermined number of intra prediction blocks based on the predetermined number of intra prediction modes; and generating a final intra prediction block of the current block by weighted fusion of the predetermined number of intra prediction blocks.

[0008] According to another embodiment of the present disclosure, a method for encoding a current block performed by an image encoding device comprises the steps of: configuring a candidate list of the current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; deriving a predetermined number of intra prediction mode candidates from the intra prediction mode candidates in the candidate list to obtain a predetermined number of intra prediction modes of the current block; generating a predetermined number of intra prediction blocks based on the predetermined number of intra prediction modes; and generating a final intra prediction block of the current block by weighted fusion of the predetermined number of intra prediction blocks.

[0009] According to another embodiment of the present disclosure, a method for providing video data to a video decoder comprises: a step of encoding the video data into a bitstream; and a step of transmitting the bitstream to the video decoder, wherein the step of encoding the video data comprises: a step of configuring a candidate list of a current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; a step of deriving a predetermined number of intra prediction mode candidates from the intra prediction mode candidates in the candidate list to obtain a predetermined number of intra prediction modes of the current block; a step of generating a predetermined number of intra prediction blocks based on the predetermined number of intra prediction modes; and a step of generating a final intra prediction block of the current block by weighted fusion of the predetermined number of intra prediction blocks.

[0010] As described above, by providing a video encoding / decoding method and device according to the present embodiment, and a recording medium storing a bitstream generated by the video encoding method / device, it is possible to improve video encoding efficiency and video quality.

[0011] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure.

[0012] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0013] FIGS. 3a and 3b are diagrams showing a plurality of intra prediction modes including wide-angle intra prediction modes.

[0014] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0015] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure.

[0016] FIG. 6 is an exemplary diagram showing a reference line of a surrounding restored area according to one embodiment of the present disclosure.

[0017] FIG. 7 is an exemplary diagram showing a peripheral pixel area according to one embodiment of the present disclosure.

[0018] FIG. 8 is a flowchart illustrating a method for weighted fusion of intra-prediction blocks according to one embodiment of the present disclosure.

[0019] FIG. 9 is a flowchart illustrating a method for constructing a candidate list based on information of a reference block according to one embodiment of the present disclosure.

[0020] Hereinafter, embodiments of the present invention will be described in detail with reference to the exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing these embodiments, if it is determined that a detailed description of related known components or functions could obscure the essence of these embodiments, such detailed description is omitted.

[0021] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image encoding device and its sub-components will be described with reference to FIG. 1.

[0022] The video encoding device may be configured to include a picture splitting unit (110), a prediction unit (120), a subtractor (130), a conversion unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse conversion unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0023] Each component of the video encoding device may be implemented in hardware or software, or as a combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the software function corresponding to each component.

[0024] A single image (video) consists of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). And each CTU is divided into one or more Coding Units (CUs) by a tree structure. Information applicable to each CU is encoded as the syntax of the CU, and information applicable to all CUs included in a single CTU is encoded as the syntax of the CTU. Additionally, information applicable to all blocks within a single slice is encoded as the syntax of the slice header, and information applicable to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). Also, information commonly referenced by one or more SPSs is encoded in a Video Parameter Set (VPS). Additionally, information commonly applicable to a single tile or tile group may be encoded as the syntax of a tile or tile group header. The syntax included in the SPS, PPS, slice header, and tile or tile group header may be referred to as high-level syntax.

[0025] The picture splitting unit (110) determines the size of the CTU. Information regarding the size of the CTU (CTU size) is encoded as a syntax of SPS or PPS and transmitted to an image decoding device.

[0026] The picture division unit (110) divides each picture constituting the image into multiple CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. The leaf nodes in the tree structure become the CUs, which are the basic units of encoding.

[0027] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of equal size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that combines two or more of these QT, BT, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT combined may be referred to as an MTT (Multiple-Type Tree).

[0028] Figure 2 is a diagram illustrating a method for dividing blocks using a QTBTTT structure.

[0029] As illustrated in FIG. 2, the CTU can first be split into a QT structure. Quadtree splitting can be repeated until the size of the splitting block reaches the minimum block size of the leaf node allowed in QT (MinQTSize). A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder (155) and signaled to the image decoder. If the leaf node of the QT is not larger than the maximum block size of the root node allowed in BT (MaxBTSize), it can be further split into one or more of the BT structure or TT structure. In the BT structure and / or TT structure, multiple splitting directions may exist. For example, there may be two directions in which the block of the corresponding node is split horizontally and vertically. As shown in Figure 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether the nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or the splitting type (binary or ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0030] Alternatively, prior to encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the value of the CU split flag (split_cu_flag) indicates that it is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a coding unit (CU), which is the basic unit of encoding. If the value of the CU split flag (split_cu_flag) indicates that it is split, the video encoding device starts encoding from the first flag in the manner described above.

[0031] When QTBT is used as another example of a tree structure, there may be two types: a type that divides the block of the corresponding node horizontally into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that divides it vertically (i.e., symmetric vertical splitting). A splitting flag (split_flag) indicating whether each node of the BT structure is split into a block of a lower layer and splitting type information indicating the type of splitting are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may also be an additional type that divides the block of the corresponding node into two blocks of an asymmetric shape. The asymmetric shape may include a shape that divides the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that divides the block of the corresponding node diagonally.

[0032] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from a CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of QTBTTT) is referred to as the 'current block'. Depending on the adoption of QTBTTT partitioning, the shape of the current block may be not only square but also rectangular.

[0033] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0034] Generally, current blocks within a picture can each be predictively coded. Typically, the prediction of a current block can be performed using an intra-prediction technique (using data from the picture containing the current block) or an inter-prediction technique (using data from a picture coded prior to the picture containing the current block). Inter-prediction includes both unidirectional and bidirectional prediction.

[0035] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture containing the current block. Multiple intra prediction modes exist depending on the prediction direction. For example, as shown in FIG. 3a, multiple intra prediction modes may include two non-directional modes, including Planar mode and DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently for each prediction mode.

[0036] For efficient directional prediction for a rectangular current block, directional modes (intra-prediction modes 67 through 80 and -1 through -14) illustrated by dashed arrows in FIG. 3b may be additionally used. These may be referred to as "wide angle intra-prediction modes." In FIG. 3b, the arrows indicate corresponding reference samples used for prediction and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. Among the wide angle intra-prediction modes, some wide angle intra-prediction modes available for the current block may be determined by the ratio of the width to the height of the rectangular current block. For example, wide-angle intra-prediction modes with an angle less than 45 degrees (intra-prediction modes 67 to 80) are available when the current block is a rectangular shape with a height less than the width, and wide-angle intra-prediction modes with an angle greater than -135 degrees (intra-prediction modes -1 to -14) are available when the current block is a rectangular shape with a width greater than the height.

[0037] The intra prediction unit (122) can determine the intra prediction mode to use for encoding the current block. In some examples, the intra prediction unit (122) may encode the current block using several intra prediction modes and select an appropriate intra prediction mode to use from the tested modes. For example, the intra prediction unit (122) may calculate the rate-distortion values ​​using a rate-distortion analysis of several tested intra prediction modes and select the intra prediction mode having the best rate-distortion features among the tested modes.

[0038] The intra prediction unit (122) selects one intra prediction mode among a plurality of intra prediction modes and predicts the current block using a calculation formula and surrounding pixels (reference pixels) determined according to the selected intra prediction mode. Information regarding the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0039] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for the block most similar to the current block within a reference picture that is encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, it generates a motion vector (MV) corresponding to the displacement between the current block in the current picture and the prediction block in the reference picture. Generally, motion estimation is performed on the lumina component, and the motion vector calculated based on the lumina component is used for both the lumina component and the chroma component. Motion information including information about the reference picture used to predict the current block and information about the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0040] The inter prediction unit (124) may perform interpolation on a reference picture or reference block to increase the accuracy of the prediction. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including those two integer samples. When the process of searching for the block most similar to the current block is performed for the interpolated reference picture, the motion vector can be expressed with precision in fractional units rather than precision in integer sample units. The precision or resolution of the motion vector can be set differently for each unit of the target area to be encoded, such as slice, tile, CTU, CU, etc. When such Adaptive Motion Vector Resolution (AMVR) is applied, information regarding the motion vector resolution to be applied to each target area must be signaled for each target area. For example, if the target area is a CU, information regarding the motion vector resolution applied to each CU is signaled. The information regarding the motion vector resolution may be information indicating the precision of the difference motion vector described later.

[0041] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors representing the block location most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from the reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and generates a first reference block and a second reference block by searching for a block similar to the current block within each reference picture. Then, it generates a prediction block for the current block by averaging or weighting the first reference block and the second reference block. Then, it transmits motion information containing information about the two reference pictures used to predict the current block and information about the two motion vectors to the entropy encoding unit (155). Here, reference picture list 0 consists of restored pictures that are prior to the current picture in the display order, and reference picture list 1 may consist of restored pictures that are prior to the current picture in the display order. However, this is not necessarily limited to this, and restored pictures prior to the current picture in the display order may be additionally included in reference picture list 0, and conversely, restored pictures prior to the current picture may be additionally included in reference picture list 1.

[0042] Various methods can be used to minimize the amount of bits required to encode motion information.

[0043] For example, if the reference picture and motion vector of the current block are identical to the reference picture and motion vector of a neighboring block, the motion information of the current block can be transmitted to an image decoder by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0044] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0045] As for the surrounding blocks for deriving merge candidates, as shown in FIG. 4, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture may be used. Additionally, a block located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as a merge candidate. For example, a block located at the same position as the current block within the reference picture (co-located block) or a block adjacent to that same position may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0046] The inter prediction unit (124) constructs a merge list containing a predetermined number of merge candidates using these surrounding blocks. Among the merge candidates included in the merge list, it selects a merge candidate to be used as movement information for the current block and generates merge index information to identify the selected candidate. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0047] Merge skip mode is a special case of merge mode; after quantization, when all transform coefficients for entropy coding are close to zero, only neighbor block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high coding efficiency can be achieved in images with minimal motion, still images, and screen content images.

[0048] Hereinafter, merge mode and merge skip mode will be collectively referred to as merge / skip mode.

[0049] Another method for encoding motion information is the AMVP (Advanced Motion Vector Prediction) mode.

[0050] In AMVP mode, the inter-prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using the surrounding blocks of the current block. As surrounding blocks used to derive predicted motion vector candidates, all or part of the left block (A0), bottom-left block (A1), top block (B0), top-right block (B1), and top-left block (B2) adjacent to the current block within the current picture shown in FIG. 4 may be used. Additionally, blocks located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) other than the current picture where the current block is located may be used as surrounding blocks to derive predicted motion vector candidates. For example, blocks located at the same position as the current block within the reference picture (co-located blocks) or blocks adjacent to the blocks at the same position may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0051] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of the surrounding blocks and determines a predicted motion vector for the current block's motion vector using the predicted motion vector candidates. Then, it calculates a difference motion vector by subtracting the predicted motion vector from the current block's motion vector.

[0052] Predicted motion vectors can be obtained by applying a predefined function (e.g., median, mean operation, etc.) to the predicted motion vector candidates. In this case, the image decoder is also aware of the predefined function. Furthermore, since the surrounding blocks used to derive the predicted motion vector candidates have already been encoded and decoded, the image decoder is also aware of the motion vectors of those surrounding blocks. Therefore, the image decoder does not need to encode information to identify the predicted motion vector candidates. Consequently, in this case, information regarding the difference motion vector and the reference picture used to predict the current block is encoded.

[0053] Meanwhile, the predicted motion vector may be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the difference motion vector and information about the reference picture used to predict the current block.

[0054] The subtractor (130) generates a residual block by subtracting the prediction block generated by the intra prediction unit (122) or the inter prediction unit (124) from the current block.

[0055] The conversion unit (140) converts residual signals within a residual block having pixel values ​​in a spatial domain into conversion coefficients in the frequency domain. The conversion unit (140) can convert the residual signals within the residual block using the entire size of the residual block as the conversion unit, or it can divide the residual block into multiple sub-blocks and use the sub-blocks as the conversion unit to perform the conversion. Alternatively, it can divide the residual signals into two sub-blocks, a conversion area and a non-conversion area, and use only the conversion area sub-block as the conversion unit to convert the residual signals. Here, the conversion area sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only the sub-block has been converted, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. Additionally, the size of the converted area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0056] Meanwhile, the transformation unit (140) can perform transformations on the residual block individually in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices may be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one pair of transformation functions with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) regarding the selected pair of transformation functions among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoder.

[0057] The quantization unit (145) quantizes the transformation coefficients output from the transformation unit (140) using quantization parameters and outputs the quantized transformation coefficients to the entropy encoding unit (155). The quantization unit (145) may quantize the associated residual block directly without transformation for any block or frame. The quantization unit (145) may apply different quantization coefficients (scaling values) depending on the position of the transformation coefficients within the transformation block. The quantization matrix applied to the quantized transformation coefficients arranged in two dimensions can be encoded and signaled to an image decoder.

[0058] The reordering unit (150) can perform reordering of coefficient values ​​for quantized residual values.

[0059] The reordering unit (150) can convert a two-dimensional coefficient array into a one-dimensional coefficient sequence using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from DC coefficients to coefficients in the high-frequency range using a zig-zag scan or a diagonal scan. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction and a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of a zig-zag scan. That is, depending on the size of the conversion unit and the intra-prediction mode, the scanning method to be used among a zig-zag scan, a diagonal scan, a vertical scan, and a horizontal scan may be determined.

[0060] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transformation coefficients output from the reordering unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0061] Additionally, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the video decoder can divide the block in the same way as the video encoding unit. Additionally, the entropy encoding unit (155) encodes information regarding a prediction type indicating whether the current block is encoded by intra prediction or by inter prediction, and encodes intra prediction information (i.e., information regarding the intra prediction mode) or inter prediction information (information regarding the encoding mode of motion information (merge mode or AMVP mode), the merge index in the case of merge mode, and the reference picture index and difference motion vector in the case of AMVP mode) according to the prediction type. Additionally, the entropy encoding unit (155) encodes information related to quantization, i.e., information regarding quantization parameters and information regarding the quantization matrix.

[0062] The inverse quantization unit (160) inversely quantizes the quantized transformation coefficients output from the quantization unit (145) to generate transformation coefficients. The inverse transformation unit (165) converts the transformation coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0063] The adder (170) restores the current block by adding the restored residual block and the prediction block generated by the prediction unit (120). The pixels within the restored current block are used as reference pixels when intra-predicting the next block in sequence.

[0064] The loop filter section (180) performs filtering on the restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. caused by block-based prediction and transformation / quantization. The loop filter section (180) may include all or part of a deblocking filter (182), a SAO (Sample Adaptive Offset) filter (184), and an ALF (Adaptive Loop Filter, 186) as an in-loop filter.

[0065] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-unit encoding / decoding, and the SAO filter (184) and ALF (186) perform additional filtering on the deblocking filtered image. The SAO filter (184) and ALF (186) are filters used to compensate for the difference between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in CTU units. In contrast, the ALF (186) performs block-unit filtering, and compensates for distortion by applying different filters by distinguishing the degree of edge and change of the corresponding block. Information regarding the filter coefficients to be used in the ALF can be encoded and signaled to an image decoder.

[0066] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in memory (190). Once all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0067] The video encoding device can store the bitstream of encoded video data on a non-transient recording medium or transmit it to a video decoding device using a communication network.

[0068] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the technologies of the present disclosure. Hereinafter, the image decoding device and its sub-components will be described with reference to FIG. 5.

[0069] The image decoding device may be configured to include an entropy decoding unit (510), a reordering unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0070] Similar to the image encoding device of FIG. 1, each component of the image decoding device may be implemented in hardware or software, or in combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

[0071] The entropy decoding unit (510) determines the current block to be decoded by decoding the bitstream generated by the video encoding device and extracting information related to block division, and extracts prediction information, information on residual signals, etc., necessary to restore the current block.

[0072] The entropy decoding unit (510) extracts information about the CTU size from the SPS (Sequence Parameter Set) or PPS (Picture Parameter Set) to determine the size of the CTU and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and divides the CTU using the tree structure by extracting division information for the CTU.

[0073] For example, when splitting a CTU using a QTBTTT structure, first, a first flag (QT_split_flag) related to QT splitting is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a second flag (mtt_split_flag) related to MTT splitting and splitting direction (vertical / horizontal) and / or splitting type (binary / ternary) information are extracted to split the corresponding leaf nodes into an MTT structure. Accordingly, each node below the leaf nodes of QT is recursively split into a BT or TT structure.

[0074] As another example, when splitting a CTU using the QTBTTT structure, a CU splitting flag (split_cu_flag) indicating whether to split the CU is first extracted, and if the block is split, a first flag (QT_split_flag) is extracted. During the splitting process, each node may undergo zero or more iterative MTT splittings after zero or more iterative QT splittings. For example, the CTU may undergo MTT splitting immediately, or conversely, only multiple QT splittings may occur.

[0075] As another example, when splitting a CTU using a QTBT structure, a first flag (QT_split_flag) related to the splitting of QT is extracted to split each node into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and split direction information are extracted.

[0076] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded using the division of the tree structure, it extracts information regarding the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-predicted, the entropy decoding unit (510) extracts syntax elements for the intra-predicted information (intra-predicted mode) of the current block. If the prediction type information indicates inter-predicted, the entropy decoding unit (510) extracts syntax elements for the inter-predicted information, namely information indicating the motion vector and the reference picture that the motion vector refers to.

[0077] Additionally, the entropy decoding unit (510) extracts information regarding quantization-related information and information regarding residual signals, as well as information regarding the quantized transformation coefficients of the current block.

[0078] The reordering unit (515) can change the sequence of one-dimensional quantized transformation coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning order performed by the image encoding device.

[0079] The inverse quantization unit (520) inversely quantizes the quantized transformation coefficients and inversely quantizes the quantized transformation coefficients using quantization parameters. The inverse quantization unit (520) may apply different quantization coefficients (scaling values) to the quantized transformation coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transformation coefficients.

[0080] The inverse transformation unit (530) generates a residual block for the current block by inversely transforming the inversely quantized transformation coefficients from the frequency domain to the spatial domain and restoring the residual signals.

[0081] Additionally, when the inverse transformation unit (530) inversely transforms only a part of the transformation block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transformation block has been transformed, information on the directionality (vertical / horizontal) of the sub-block (cu_sbt_horizontal_flag) and / or information on the position of the sub-block (cu_sbt_pos_flag), restores residual signals by inversely transforming the transformation coefficients of the corresponding sub-block from the frequency domain to the spatial domain, and creates a final residual block for the current block by filling the areas that have not been inversely transformed with "0" values ​​of residual signals.

[0082] Additionally, when MTS is applied, the inverse transformation unit (530) determines a transformation function or transformation matrix to be applied in the horizontal and vertical directions, respectively, using MTS information (mts_idx) signaled from the video encoding device, and performs an inverse transformation on the transformation coefficients within the transformation block in the horizontal and vertical directions using the determined transformation function.

[0083] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is an intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is an inter prediction.

[0084] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0085] The inter prediction unit (544) determines the motion vector of the current block and the reference picture that the motion vector refers to using the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510), and predicts the current block using the motion vector and the reference picture.

[0086] The adder (550) restores the current block by adding the residual block output from the inverse transformation unit (530) and the prediction block output from the inter prediction unit (544) or the intra prediction unit (542). The pixels within the restored current block are used as reference pixels when intra-predicting the block to be decoded later.

[0087] The loop filter section (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as an in-loop filter. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-unit decoding. The SAO filter (564) and the ALF (566) perform additional filtering on the restored blocks after deblocking filtering to compensate for the difference between the restored pixels and the original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0088] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture to inter-predict blocks within the picture to be encoded later.

[0089] The present embodiment relates to the encoding and decoding of an image (video) as described above. More specifically, the present invention provides an image encoding / decoding method and apparatus for generating an intra-prediction block in an inter-prediction utilizing intra-prediction information, and a recording medium for storing a bitstream generated by the image encoding method / apparatus.

[0090] The following embodiments may be performed by a prediction unit (120) within a video encoding apparatus. Additionally, the following embodiments may be performed by a prediction unit (540) within a video decoding apparatus.

[0091] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization during the encoding of the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoder. The video decoder can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoder (510).

[0092] In the following description, the term 'target block' may be used interchangeably with 'current block' or 'Coding Unit (CU).' Alternatively, 'target block' may refer to a specific area of ​​a Coding Unit.

[0093] Also, a value of one flag being true indicates that the flag is set to 1. Also, a value of one flag being false indicates that the flag is set to 0.

[0094] The decoder-side includes all or part of an inverse quantizer (160), an inverse transform (165), a prediction unit (120), an adder (170), a loop filter (180), and a memory (190) in the image encoding device illustrated in FIG. 1. Alternatively, the decoder-side includes all or part of an inverse quantizer (520), an inverse transform (530), a prediction unit (540), an adder (550), a loop filter (560), and a memory (570) in the image decoding device illustrated in FIG. 5. In relation to a series of decoding processes, the decoder-side of the image encoding device and the decoder-side of the image decoding device perform the same operation. The image encoding device determines information related to the operation of the decoder-side and signals the determined information to the image decoding device. The image decoding device can decode the signaled information and operate the decoder-side based on the decoded information.

[0095] I. Inter-prediction technology utilizing intra-prediction

[0096] CIIP (Combined Inter-Intra Prediction) mode generates final predicted samples by weighting the inter-predicted samples and the intra-predicted samples. GPM (Geometric Partitioning Mode) mode divides a block into two sub-regions along a partition boundary, predicts each sub-region based on different information, and generates a final predicted block by weighting the predicted sub-regions. In this process, intra-predicted samples may be generated from one or more regions. GPM and CIIP can also be combined with IBC (Intra Block Copy), which are designated as IBC-GPM mode and IBC-CIIP mode, respectively.

[0097] In intra-prediction utilized in the inter-prediction technique described above, the video encoding device constructs an intra-prediction mode candidate list containing N (where N is an integer greater than or equal to 2) intra-prediction mode (IPM) candidates and signals an index indicating one of the candidates to the video decoder. In CIIP and GPM, the candidate list may be constructed based on Planar, a prediction mode derived according to a DIMD (decoder-side intra mode derivation) mode using recovered samples, a prediction mode derived according to a TIMD (template-based intra mode derivation) mode using recovered samples, a prediction mode derived according to an OBIC (Occurrence-based Intra Coding) mode using the frequency of prediction modes of surrounding blocks, etc. The video decoder decodes the index indicating the final candidate in the candidate list and generates intra-prediction samples based on the index.

[0098] Below, the intra prediction mode candidate list, IPM candidate list, and candidate list can be used interchangeably.

[0099] The TIMD mode, DIMD mode, and OBIC mode are described below.

[0100] The TIMD mode can use m and n restored lines surrounding the current block as templates, as shown in FIG. 6. m and n are integers greater than or equal to 0 or 1, and if both m and n are integers greater than or equal to 1, they may be identical. TIMD is a technique that derives an intra prediction mode based on the template matching cost. The TIMD mode generates a prediction template for the current block template from the surrounding restored samples of the current block template (e.g., reference lines of the restored area in FIG. 6) using intra prediction mode candidates (e.g., intra prediction modes included in the MPM list). The TIMD mode can calculate the template matching cost for the corresponding intra prediction mode by calculating the cost between the restored samples within the current block template and the prediction samples within the prediction template. The process of calculating the template cost can be referred to as Template Matching (TM).

[0101] As an example, N prediction modes (where N is an integer greater than or equal to 1) included in the MPM (Most Probable Mode) list can be used to calculate the template matching cost. The MPM list may include intra prediction modes of adjacent blocks of the current block, intra prediction modes of non-adjacent areas, and prediction modes derived by applying directional analysis to samples of a certain surrounding restored area.

[0102] The TIMD mode calculates the cost corresponding to each intra prediction mode based on the template and the reference line of the template, selects L modes (where L is an integer greater than or equal to 1) that have the minimum cost, and can use the selected modes as intra prediction mode candidates.

[0103] One or more sample lines may be used as reference lines for the template. In the example of FIG. 6, one sample line is used as a reference line.

[0104] To calculate costs, cost estimation methods such as SAD (Sum of Absolute Difference), SATD (Sum of Absolute Transformed Difference), and MR-SAD (Mean Reduced Sum of Absolute Difference) may be used. As an example, one of the aforementioned cost estimation methods may be used, or two or more methods may be used simultaneously.

[0105] As mentioned above, since L is an integer greater than or equal to 1, at least one intra prediction mode can be derived, for example, based on the template matching cost.

[0106] In TIMD mode, the final prediction block can be generated as a weighted sum between the prediction blocks of intra prediction modes derived according to template matching, and the prediction blocks based on Planar modes or block vectors. In the present disclosure, TIMD mode may refer to a process of deriving intra prediction modes, or a process that includes the generation of the final prediction block based on a weighted sum.

[0107] The DIMD mode is a method for deriving a decoder-based intra prediction mode, inducing the intra prediction mode on the decoder side based on directional analysis of recovered samples. At this time, recovered samples within the surrounding region of the current block can be used for directional analysis.

[0108] As an example, the number of sample lines used for direction analysis may be m (where m is an integer greater than or equal to 2). As the surrounding pixel area used for direction analysis, either the left or the top of the current block may be used, or both the left and top of the current block may be used. An area containing surrounding reconstructed samples may be used for direction analysis.

[0109] As shown in FIG. 7, the DIMD mode applies an edge detection filter (Sobel filter, Prewitt filter, Robert cross filter, etc.) to the surrounding pixel area to calculate the vertical and horizontal gradients between the sample and surrounding samples, and analyzes the gradients to extract angle information and amplitude information. The DIMD mode can generate a Histogram of Gradient (HoG) using the angle information and amplitude information. The angle information can be substituted with a directional prediction mode on the HoG. Based on the magnitude of the directional prediction mode on the HoG, an intra prediction mode of the current block can be derived. The number of prediction modes derived based on the HoG can be L (where L is an integer greater than or equal to 1). For example, the two prediction modes with the largest magnitudes can be derived as the intra prediction modes of the current block.

[0110] In DIMD mode, the final prediction block can be generated as a weighted sum between the prediction blocks of intra prediction modes derived according to directional analysis, and the prediction blocks based on Planar modes or block vectors. In the present disclosure, DIMD mode may refer to a process of deriving intra prediction modes, or a process that includes the generation of the final prediction block based on a weighted sum.

[0111] The OBIC mode derives multiple intra-predicted directions based on prediction information of spatially adjacent and non-adjacent blocks in the restoration area around the current block, generates corresponding prediction blocks using one or more derived intra-predicted directions, and generates the final prediction block of the current block by weighting the prediction blocks.

[0112] In OBIC mode, to induce an intra prediction mode, a candidate list of intra prediction modes is constructed based on prediction information of adjacent and non-adjacent blocks, and a histogram of occurrence (Histogram of occurrence, HoC, hereinafter used interchangeably with histogram of occurrence frequency) is constructed based on the intra prediction direction and block size of the blocks present in the candidate list. By weighting the prediction blocks generated according to Planar mode or Block Vector (BV)-based prediction blocks, and up to five directional prediction modes selected in order of increasing cumulative value in the HoC, the OBIC mode generates a final prediction block.

[0113] The OBIC mode may use blocks predicted by the intra prediction mode among adjacent blocks to construct the candidate list. For example, blocks predicted according to DIMD, OBIC, SGPM (Spatial Geometry Partitioning Mode), TIMD, etc., and blocks predicted according to the intra prediction mode may be used. The candidate list may contain up to 31 candidate blocks (hereinafter used interchangeably with candidates). Duplicate candidates are removed during the process of constructing the candidate list. The candidate list may be reordered based on the distance between the current block and the candidate blocks. Finally, the candidate list is constructed using only up to 20 candidates that are closest in distance.

[0114] The OBIC mode constructs a Higher-Order Community (HoC) using the intra-prediction modes and block sizes of the blocks in the candidate list. The histogram size is accumulated for the intra-prediction modes of the candidate blocks, and the accumulated value is equal to the size of the candidate blocks. In cases where candidate blocks are generated according to multiple intra-prediction modes, such as DIMD, OBIC, SGPM, and TIMD, the histogram size is accumulated by the size of the candidate blocks for all intra-prediction modes. The HoC is constructed by repeatedly performing the aforementioned process for all candidate blocks.

[0115] The OBIC mode selects up to five prediction modes in order of increasing cumulative value from the HoC and determines the weight of each prediction mode based on the cumulative value. A final prediction block is generated by weighting the up to five prediction blocks generated according to the selected prediction modes, and the prediction blocks generated based on Planar mode or BV. At this time, a prediction block based on Planar mode or BV mode may be selected based on the template costs of the left and top regions of the restored current block. BV information may be obtained from blocks predicted according to IntraTMP or IBC (Intra Block Copy) mode among spatially adjacent or non-adjacent blocks.

[0116] In the present disclosure, the OBIC mode may mean a process of inducing an intra prediction mode, or a process that includes the generation of a final prediction block based on a weighted sum.

[0117] The following embodiments are described with reference to an image decoding device, but may be implemented identically or similarly in an image encoding device. Alternatively, the following embodiments are described with reference to the decoder side of the image decoding device, but may be implemented identically or similarly in the decoder side of an image encoding device.

[0118] II. Embodiments according to the present disclosure

[0119] As described above, the image decoder may construct an IPM candidate list including intra prediction mode candidates for inter prediction. Subsequently, when performing intra prediction for the current block, the image decoder may perform intra prediction fusion. The image decoder may generate a weighted fused prediction block using K candidates among the N candidates included in the IPM candidate list. Here, N is a positive integer greater than or equal to 2, and K is a positive integer greater than or equal to 2 and less than or equal to N. The intra prediction mode may be one of a non-directional mode (DC, Planar) and a directional prediction mode.

[0120] Hereinafter, weighted fusion can be used interchangeably with weighted sum.

[0121] The image decoding device may perform average prediction fusion when performing intra prediction fusion for the intra block to be predicted (hereinafter, current intra block). Here, average prediction fusion may be a method of averaging prediction blocks obtained according to K different prediction modes among a candidate list on a sample-by-sample basis. As previously mentioned, K is a positive integer greater than or equal to 2.

[0122] The image decoder may perform weighted average prediction fusion when performing intra prediction fusion on the current intra block. Here, weighted average prediction fusion may be a method of fusing prediction blocks obtained according to K different prediction modes among a candidate list by applying weights on a sample basis. As previously mentioned, K is a positive integer greater than or equal to 2.

[0123] For example, when K is 2, weighted fusion based on two prediction modes A and B can be performed on the current intra-block. In this case, weighted fusion is performed according to Equation 1.

[0124]

[0125] In mathematical equation 1, P A (i,j), P B (i,j) represent the prediction block according to prediction mode A and the prediction block according to prediction mode B, respectively, and P fus (i,j) represents the weighted fused prediction block. w A (i,j), w B (i,j) represent the weights corresponding to the prediction samples according to prediction mode A and prediction mode B, respectively, and w A (i,j)+w B (i,j)=1 can be satisfied.

[0126] For example, when K is 3, weighted fusion based on three prediction modes A, B, and C can be performed on the current intra-block. In this case, weighted fusion is performed according to Equation 2.

[0127]

[0128] In mathematical equation 2, P A (i,j), P B (i,j), P C (i,j) represent the prediction block according to prediction mode A, the prediction block according to prediction mode B, and the prediction block according to prediction mode C, respectively, and P fus (i,j) represents the weighted fused prediction block. w A (i,j), w B (i,j), w C (i,j) represent the weights corresponding to the prediction samples according to prediction mode A, prediction samples according to prediction mode B, and prediction samples according to prediction mode C, respectively, and w A (i,j)+w B (i,j)+w C (i,j)=1 can be satisfied.

[0129] Weights may be defined differently based on slice type (I / P / B slice), signal components (Y / U / V), template cost (i.e., error cost), etc. The template represents the surrounding restored samples of the current block and may be referred to as the current template hereinafter.

[0130] As an example, weights can be set based on the error costs of K candidates. Candidates with small error costs may be assigned large weights. In other words, the weight of each candidate can be set inversely proportional to the error cost. The error cost for each of the K candidates can be calculated, for instance, using the current template. Specifically, the error cost for each candidate can be calculated by generating a prediction template by predicting the current template for each of the K candidates, and by calculating the matching cost between the current template and the prediction template.

[0131] As another example, weights can be set to predefined values.

[0132] The intra-predictive fusion indicator indicates whether the prediction mode of the current intra-block is a fusion mode. The video encoding device can determine the intra-predictive fusion indicator and signal it to the video decoder. The video decoder can parse the intra-predictive fusion indicator and perform intra-predictive fusion based on the parsed intra-predictive fusion indicator.

[0133] Here, context-adaptive binary arithmetic coding (CABAC) may be applied to the intra-predictive fusion indicator. In this case, the probabilistic model of CABAC may be set differently based on the surrounding blocks of the current block. Alternatively, bypass coding may be applied to the intra-predictive fusion indicator.

[0134] When the aforementioned intra-prediction fusion indicator is signaled / parsed, indices indicating K candidates may be signaled / parsed. The image decoder may generate an intra-prediction block by fusing the parsed K candidates from the IPM candidate list. As another example, the image decoder may generate an intra-prediction block by fusing K predefined candidates from the IPM candidate list. The K predefined candidates may be set, for example, based on error costs. That is, K candidates with small error costs may be selected for intra-prediction fusion.

[0135] The aforementioned intra-predictive fusion indicator may not be signaled / parsed. Without signaling / parsing the intra-predictive fusion indicator, the image decoder may generate an intra-prediction block by fusing K predefined candidates from the IPM candidate list. The K predefined candidates may be set, for example, based on error costs.

[0136] As an embodiment according to the present disclosure, the image decoding device may construct a candidate list based on information of an intra-predicted reference block in an inter-prediction that requires intra-predicted samples, without constructing the aforementioned IPM candidate list. Subsequently, the image decoding device may generate an intra-predicted block based on the constructed candidate list.

[0137] Hereinafter, the intra-predicted reference block is referred to as the reference intra-block candidate. The candidate list constructed based on the information of the intra-predicted reference block is referred to as the reference intra-block candidate list.

[0138] First, describe the steps for constructing the reference intra-block candidate list.

[0139] The video decoder may include at least one or up to L (where L is a positive integer) of the surrounding intra prediction blocks spatially and temporally adjacent to the current block in the reference intra block candidate list. Here, L may be a preset value according to an agreement between the video encoder and the video decoder. Alternatively, L may be a value signaled from the video encoder to the video decoder.

[0140] The image decoding device can construct a reference intra-block candidate list using at least one of the following methods or a combination of at least one method.

[0141] The image decoding device may utilize prediction information of the current block and adjacent blocks. Among the surrounding blocks adjacent to the current block, up to L may be included in the reference intra-block candidate list for the prediction of the current intra-block. An adjacent block being adjacent to the current block means that at least one of the boundaries and vertices of the current block touches at least one of the boundaries and vertices of the corresponding surrounding block.

[0142] The image decoding device may utilize the size of a neighboring block adjacent to the current block. Based on the size of the neighboring blocks adjacent to the current block, up to L neighboring blocks may be included in the reference intra-block candidate list. For example, up to L neighboring blocks with a size of M×N or greater may be included in the reference intra-block candidate list. Alternatively, up to L neighboring blocks with a size of M×N or less may be included in the reference intra-block candidate list.

[0143] The video decoding device can utilize surrounding blocks that exist within a specific distance from the current block. Among the surrounding blocks that exist within a specific distance from the location of the current block, up to L can be included in the reference intra-block candidate list.

[0144] The image decoding device may use a reference block indicated by a merge candidate of the current block. If a sample at a specific location within the reference block indicated by the merge candidate is intra-predicted, the corresponding block at that specific location may be included in the reference intra-block candidate list. Here, the corresponding block at the specific location represents a block containing the sample at the specific location within the reference block.

[0145] The image decoding device may use the surrounding blocks of the reference block indicated by the merge candidate of the current block. If the surrounding blocks of the reference block indicated by the merge candidate are intra-predicted, the corresponding surrounding blocks may be included in the reference intra-block candidate list.

[0146] In the reference intra-block candidate list configured as described above, an index indicating the finally referenced block, i.e., the reference intra-block of the current intra-block, can be signaled / parsed. Alternatively, in the reference intra-block candidate list, for example, a single candidate at a specific location may be fixed, and the fixed single candidate may be used as the reference intra-block of the current intra-block.

[0147] Next, the step of generating an intra prediction block based on the configured list of reference intra block candidates is described.

[0148] The image decoder can generate an intra prediction block based on prediction information of a reference intra block derived from a list of reference intra block candidates. Here, the prediction information may include all prediction techniques performed in intra prediction, such as non-directional mode (DC, Planar), directional mode, IntraTMP, IBC, MIP (Matrix-based intra prediction), OBIC, MRL (multiple reference line), TIMD, DIMD, EIP (extrapolation filter-based intra prediction), fusion-based prediction mode, ISP (Intra sub-partitioning), TMRL (template-based MRL), etc.

[0149] As an example, if the reference intra block has generated prediction samples based on a specific technique among the intra prediction techniques, the current intra block can generate prediction samples according to the aforementioned specific technique.

[0150] If the reference intra-block is predicted according to Planar mode, the current intra-block can generate predicted samples according to Planar mode.

[0151] If the reference intra-block is predicted according to IntraTMP, the current intra-block can generate predicted samples by copying samples at locations indicated by the block vector of the reference intra-block.

[0152] When a reference intra-block is predicted according to DIMD, the current intra-block can perform the DIMD technique to induce multiple directional modes and generate multiple prediction blocks based on the induced directional modes. Subsequently, a final intra-prediction block can be generated by fusing multiple prediction blocks and non-directional prediction blocks. The non-directional prediction blocks may be prediction blocks generated based on Planar mode or BV.

[0153] As an embodiment according to the present disclosure, the image decoding device may correct intra-predicted samples when performing intra-predicting of the current intra-block. The intra-predicted block may be generated based on the IPM candidate list or reference intra-block candidate list as described above. Correction of the intra-predicted samples may mean post-processing of the intra-predicted result of the current block.

[0154] Correction of intra-predicted samples can be performed using at least one of correction methods such as bilateral filtering, boundary smoothing, PDPC (Position Dependent Prediction Combination), and gradient PDPC.

[0155] PDPC technology corrects an initial predicted pixel generated using the values ​​of surrounding pixels according to the intra-prediction mode by performing a weighted sum of the surrounding pixels in the opposite direction and the predicted pixel based on the distance between the predicted pixel and the surrounding pixels in the opposite direction, when the location of the predicted pixel and the location of the surrounding pixels used are far apart. In this case, the closer the distance between the surrounding pixels in the opposite direction and the predicted pixel, the greater the weight of the surrounding pixels in the opposite direction. PDPC technology can be applied to directional prediction modes where surrounding pixels in the opposite direction are available.

[0156] When the directional mode is horizontal mode (No. 18) or vertical mode (No. 50), a gradient-based approach may be used because neighboring pixels in the opposite direction are unavailable. In the case of horizontal mode, the predicted pixel is corrected by weighted combination of the difference between the r(-1,-1) sample and the r(x,-1) sample and the predicted pixel p(x,y). The r(-1,-1) sample and the r(x,-1) sample represent the top reference samples of the prediction block, and the difference between the r(-1,-1) sample and the r(x,-1) sample corresponds to the gradient in the horizontal direction. In the case of vertical mode, the predicted pixel is corrected by weighted combination of the difference between the r(-1,-1) sample and the r(-1, y) sample and the predicted pixel p(x,y). The r(-1,-1) sample and the r(-1, y) sample represent the left reference samples of the prediction block, and the difference between the r(-1,-1) sample and the r(-1, y) sample corresponds to the vertical gradient.

[0157] In cases where the directional prediction mode is not horizontal or vertical but neighboring pixels in the opposite direction cannot be used, gradient PDPC uses a gradient-based approach as described above. Depending on the position of the predicted pixel, the direction of the prediction mode, and / or the size of the prediction block, the gradient can be defined as the difference between the filtered reference sample and the unfiltered reference sample. For example, if the directional prediction mode is close to the vertical direction but neighboring pixels in the opposite direction cannot be used, the unfiltered reference sample is defined as r(-1, y), and the filtered reference sample located at the top of the current block is defined as r(-1+d, -1). If the directional prediction mode is close to the horizontal direction but neighboring pixels in the opposite direction cannot be used, the unfiltered reference sample is defined as r(x, -1), and the filtered reference sample located to the left of the current block is defined as r(-1, -1+d). Here, d is used to indicate the filtered position. Gradient PDPC can correct predicted pixels by adding the predicted pixels and gradients in a weighted sum.

[0158] The image decoder can determine whether to correct intra-predicted samples based on the intra-prediction mode of the current intra-block. For example, if the intra-prediction mode of the current intra-block is an intra-prediction fusion mode, the image decoder can perform correction of the intra-predicted samples. Alternatively, if the intra-prediction mode of the current intra-block is an intra-prediction fusion mode, the image decoder may not perform correction of the intra-predicted samples.

[0159] Hereinafter, using the illustration of FIG. 8, a method for weighted fusion of intra prediction blocks in an inter prediction technique utilizing intra prediction is described. The illustration of FIG. 8 can be performed by an image encoding device and an image decoding device. FIG. 8 is described based on the image encoding device, and if necessary, the operation by the image decoding device is additionally described.

[0160] FIG. 8 is a flowchart illustrating a method for weighted fusion of intra-prediction blocks according to one embodiment of the present disclosure.

[0161] The video encoding device can obtain a flag indicating whether to weight-fuse a preset number of intra-prediction blocks from a high level. Alternatively, the video encoding device can determine the flag in terms of rate distortion optimization. The video encoding device can encode the flag. The video decoder can decode the flag from the bitstream.

[0162] When a flag indicates weighted fusion of a preset number of intra-prediction blocks, the image encoding device may perform the following steps.

[0163] The video encoding device constructs a candidate list for the current block (S800). Here, the candidate list includes intra prediction mode candidates. In an inter prediction technique utilizing intra prediction, the video encoding device may use the candidate list for intra prediction of the current block.

[0164] The video encoding device derives a preset number of intra prediction mode candidates from the intra prediction mode candidates in the candidate list and obtains a preset number of intra prediction modes of the current block (S802).

[0165] As an example, the video encoding device may obtain indices indicating a preset number of intra prediction mode candidates from an upper level. Alternatively, the video encoding device may determine the indices in terms of rate distortion optimization. The video encoding device may encode the indices. The video decoder may decode the indices from the bitstream.

[0166] The video encoding device can obtain a preset number of intra prediction modes from a candidate list based on indices.

[0167] As another example, a video encoding device can acquire a preset number of intra prediction mode candidates based on an error cost. The error cost is a matching cost based on the difference between the current template and the prediction template. The prediction template can be generated by predicting the current template according to each intra prediction mode candidate among the preset number of intra prediction mode candidates.

[0168] The video encoding device generates a preset number of intra prediction blocks based on a preset number of intra prediction modes (S804).

[0169] The video encoding device generates the final intra prediction block of the current block by weighting and fusing a preset number of intra prediction blocks (S806).

[0170] The video encoding device can set weights for a preset number of intra prediction blocks based on an error cost, and can weight-fuse the preset number of intra prediction blocks based on the set weights. The error cost is a matching cost based on the difference between the current template and the prediction template. The prediction template can be generated by predicting the current template according to each intra prediction mode among a preset number of intra prediction modes.

[0171] The video encoding device can set the weights of a preset number of intra prediction blocks to predefined values ​​and weight-fuse the preset number of intra prediction blocks based on the set weights.

[0172] The video encoding device can correct prediction samples within the final intra-prediction block using at least one of a plurality of correction methods.

[0173] The video encoding device generates an inter-prediction block of the current block (S808).

[0174] For example, a video encoding device can generate inter-prediction blocks according to inter-prediction techniques based on CIIP, GPM, etc.

[0175] The video encoding device generates the final prediction block of the current block based on the final intra prediction block and the inter prediction block (S810).

[0176] Meanwhile, if the flag does not indicate weighted fusion of a preset number of intra prediction blocks, for example, the video encoding device may derive one intra prediction mode candidate from a candidate list to obtain the intra prediction mode of the current block. The video encoding device may generate the intra prediction block of the current block based on the obtained intra prediction mode. The video encoding device may generate the final prediction block of the current block based on the intra prediction block and the inter prediction block.

[0177] Subsequently, the video encoding device can generate a residual block by subtracting the final prediction block from the current block. The video encoding device can generate transformation coefficients by applying transformation / quantization to the residual block and encode the generated transformation coefficients.

[0178] The image decoder can decode the quantized transform coefficients of the current block from the bitstream and generate a residual block by applying inverse quantization / inverse transform to the quantized transform coefficients. The image decoder can restore the current block by adding the final prediction block and the residual block.

[0179] Hereinafter, using the illustration of FIG. 9, a method for constructing a candidate list based on the information of an intra-predicted reference block in an inter-prediction technique utilizing intra-prediction is described. The illustration of FIG. 9 can be performed by an image encoding device and an image decoding device. FIG. 9 is described based on the image encoding device, and if necessary, the operation by the image decoding device is additionally described.

[0180] FIG. 9 is a flowchart illustrating a method for constructing a candidate list based on information of a reference block according to one embodiment of the present disclosure.

[0181] The video encoding device obtains an index indicating one of the reference intra-block candidates of the current block (S900).

[0182] The video encoding device can, for example, obtain an index from a high level. As another example, the video encoding device can determine the index in terms of rate distortion optimization. The video encoding device can encode the index. The video decoder can decode the index from a bitstream.

[0183] The video encoding device constructs a candidate list for the current block (S902). Here, the candidate list (i.e., the reference intra-block candidate list) includes reference intra-block candidates for the current block. In an inter-prediction technique utilizing intra-prediction, the video encoding device can use the candidate list for intra-prediction of the current block.

[0184] The video encoding device may include at least one or up to L (where L is a positive integer) of the surrounding intra-prediction blocks that are spatially / temporally adjacent to the current block in the candidate list.

[0185] The video encoding device can construct a candidate list based on prediction information of the current block and adjacent blocks, the size of adjacent blocks, the distance between the current block and surrounding blocks, a reference block indicated by a merge candidate of the current block, surrounding blocks of the reference block, etc.

[0186] The video encoding device derives a reference intra-block of the current block from a candidate list based on an index, and sets an intra-prediction technique of the current block based on the prediction information of the reference intra-block (S904).

[0187] The video encoding device generates an intra prediction block of the current block based on the intra prediction technology of the current block (S906).

[0188] As an example, if the reference intra block generates a prediction block based on a specific technique among intra prediction techniques, the image decoder can generate a prediction block of the current block according to the aforementioned specific technique.

[0189] The video encoding device can correct prediction samples within an intra-prediction block using at least one of a plurality of correction methods.

[0190] The video encoding device generates an inter-prediction block of the current block (S908).

[0191] For example, a video encoding device can generate inter-prediction blocks according to inter-prediction techniques based on CIIP, GPM, etc.

[0192] The video encoding device generates a final prediction block of the current block based on an intra prediction block and an inter prediction block (S910).

[0193] Subsequently, the video encoding device can generate a residual block by subtracting the final prediction block from the current block. The video encoding device can generate transformation coefficients by applying transformation / quantization to the residual block and encode the generated transformation coefficients.

[0194] The image decoder can decode the quantized transform coefficients of the current block from the bitstream and generate a residual block by applying inverse quantization / inverse transform to the quantized transform coefficients. The image decoder can restore the current block by adding the final prediction block and the residual block.

[0195] Although the flowcharts and timing diagrams in this specification describe each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present disclosure. In other words, a person skilled in the art to which one embodiment of the present disclosure belongs may modify and adapt the flowcharts and timing diagrams in various ways, such as changing the order described in the flowcharts and timing diagrams or executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present disclosure; therefore, the flowcharts and timing diagrams are not limited to a chronological order.

[0196] It should be understood that the exemplary embodiments described above may be implemented in many different ways. The functions or methods described in one or more examples may be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "...unit" to particularly emphasize their implementation independence.

[0197] Meanwhile, the various functions or methods described in the present embodiment may be implemented as instructions stored in a non-transient recording medium that can be read and executed by one or more processors. A non-transient recording medium includes, for example, any type of recording device in which data is stored in a form readable by a computer system. For example, a non-transient recording medium includes storage media such as an EPROM (erasable programmable read-only memory), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0198] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

[0199]

[0200] CROSS-REFERENCE TO RELATED APPLICATION

[0201] This patent application claims priority to Korean patent application No. 10-2024-0143671 filed on October 21, 2024, the entire contents of which are incorporated into this patent application by reference.

Claims

1. A method for restoring a current block performed by an image decoding device, A step of constructing a candidate list of the current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; A step of deriving a preset number of intra prediction mode candidates from the intra prediction mode candidates in the above candidate list to obtain a preset number of intra prediction modes of the above current block; A step of generating a predetermined number of intra prediction blocks based on the above-mentioned predetermined number of intra prediction modes; and A step of generating the final intra prediction block of the current block by weightedly fusing the aforementioned preset number of intra prediction blocks. A method including 2. In Paragraph 1, The step of generating an inter-prediction block of the above current block; and A step of generating the final prediction block of the current block based on the final intra prediction block and the inter prediction block. A method that further includes.

3. In Paragraph 1, It further includes the step of decoding a flag indicating whether the above-mentioned preset number of intra-prediction blocks are weightedly fused, and A method in which, when the above flag indicates weighted fusion of the above-predicted number of intra prediction blocks, the above-predicted number of intra prediction modes are obtained.

4. In Paragraph 1, The step of acquiring the above-mentioned preset number of intra prediction modes is, A step of decoding indices indicating the aforementioned preset number of intra prediction mode candidates; and A step of obtaining a predetermined number of intra prediction modes from the candidate list based on the above indices A method including 5. In Paragraph 1, The step of acquiring the above-mentioned preset number of intra prediction modes is, Acquire the above-mentioned preset number of intra prediction mode candidates based on error costs, The above error cost is a matching cost based on the difference between the current template and the prediction template, and the prediction template is generated by predicting the current template according to each intra prediction mode candidate among the above-set number of intra prediction mode candidates.

6. In Paragraph 1, The step of generating the above-mentioned final intra-prediction block is, Based on the error cost, weights are set for the above-mentioned preset number of intra prediction blocks, and based on the above-mentioned weights, the above-mentioned preset number of intra prediction blocks are weightedly fused, The above error cost is a matching cost based on the difference between the current template and the predicted template, and the predicted template is generated by predicting the current template according to each intra prediction mode among the above-set number of intra prediction modes.

7. In Paragraph 1, The step of generating the above-mentioned final intra-prediction block is, A method of setting the weights of the above-mentioned number of intra prediction blocks to predefined values ​​and weight-fusing the above-mentioned number of intra prediction blocks based on the above-mentioned weights.

8. In Paragraph 1, A method further comprising the step of correcting prediction samples within the final intra-prediction block.

9. A method for encoding a current block performed by an image encoding device, A step of constructing a candidate list of the current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; A step of deriving a preset number of intra prediction mode candidates from the intra prediction mode candidates in the above candidate list to obtain a preset number of intra prediction modes of the above current block; A step of generating a predetermined number of intra prediction blocks based on the above-mentioned predetermined number of intra prediction modes; and A step of generating the final intra prediction block of the current block by weightedly fusing the aforementioned preset number of intra prediction blocks. A method including 10. In Paragraph 9, The step of generating an inter-prediction block of the above current block; and A step of generating the final prediction block of the current block based on the final intra prediction block and the inter prediction block. A method that further includes.

11. In Paragraph 9, A step of obtaining a flag indicating whether the above-mentioned preset number of intra-prediction blocks are weightedly fused; and Step of encoding the above flag Includes more, A method in which, when the above flag indicates weighted fusion of the above-predicted number of intra prediction blocks, the above-predicted number of intra prediction modes are obtained.

12. In Paragraph 9, The step of acquiring the above-mentioned preset number of intra prediction modes is, A step of obtaining indices indicating the aforementioned preset number of intra prediction mode candidates; and A step of obtaining a predetermined number of intra prediction modes from the candidate list based on the above indices A method including 13. A method for providing video data to a video decoder, A step of encoding the above video data into a bitstream; and Step of transmitting the above bitstream to the above video decoder Includes, The step of encoding the above video data is, A step of constructing a candidate list of the current block, wherein the candidate list includes intra prediction mode candidates and is used for intra prediction of the current block in an inter prediction technique utilizing intra prediction; A step of deriving a preset number of intra prediction mode candidates from the intra prediction mode candidates in the above candidate list to obtain a preset number of intra prediction modes of the above current block; A step of generating a predetermined number of intra prediction blocks based on the above-mentioned predetermined number of intra prediction modes; and A step of generating the final intra prediction block of the current block by weightedly fusing the aforementioned preset number of intra prediction blocks. A method including

Images