Method for generating adaptive histogram of occurrence of intra prediction mode using weight

Abstract

Disclosed is a method for adaptively generating a histogram of occurrence of an intra prediction mode on the basis of a weight. In the present embodiment, an image decoding device generates a histogram of occurrence indicating an occurrence frequency of intra prediction modes of previously reconstructed blocks. The image decoding device selects at least one intra prediction mode of the current block on the basis of the histogram of occurrence. The image decoding device predicts the current block according to the at least one intra prediction mode. The image decoding device derives the occurrence frequency of the intra prediction modes of the previously reconstructed blocks on the basis of the distance between each of the previously reconstructed blocks and the current block.

Description

A method for generating adaptive generation histograms of intra prediction modes using weights

[0001] The present disclosure relates to a video encoding / decoding method, device, and recording medium for storing a bitstream, and more particularly, to a method for adaptively generating an occurrence histogram of an intra prediction mode based on weights in occurrence-based prediction.

[0002] The content described below merely provides background information related to the present invention and does not constitute prior art.

[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.

[0004] Therefore, when storing or transmitting video data, the encoder compresses the video data and stores or transmits it, and the decoder receives the compressed video data, decompresses it, and plays it back. These video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.

[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.

[0006] The present disclosure aims to provide a video encoding / decoding method and device that adaptively generate a histogram of occurrence (HoC) using weights according to an intra prediction direction and weights according to a distance between blocks, and a recording medium that stores a bitstream generated by the video encoding method / device.

[0007] According to an embodiment of the present disclosure, a method for restoring a current block, performed by an image decoding device, comprises the steps of: generating a histogram of occurrence indicating an occurrence frequency of intra prediction modes of restored blocks, wherein the restored blocks include at least one adjacent block and at least one non-adjacent block of the current block; selecting at least one intra prediction mode of the current block based on the occurrence histogram; and predicting the current block according to the at least one intra prediction mode, wherein the step of generating the occurrence histogram includes the step of deriving an occurrence frequency for intra prediction modes of the restored blocks based on a distance between each of the restored blocks and the current block.

[0008] According to another embodiment of the present disclosure, a method for encoding a current block, performed by a video encoding apparatus, comprises: generating a histogram of occurrence indicating an occurrence frequency of intra prediction modes of restored blocks, wherein the restored blocks include at least one adjacent block and at least one non-adjacent block of the current block; selecting at least one intra prediction mode of the current block based on the occurrence histogram; and predicting the current block according to the at least one intra prediction mode, wherein the generating of the occurrence histogram includes a step of deriving an occurrence frequency for intra prediction modes of the restored blocks based on a distance between each of the restored blocks and the current block.

[0009] According to another embodiment of the present disclosure, a method for providing video data to a video decoding device is provided, comprising: encoding the video data into a bitstream; and transmitting the bitstream to the video decoding device, wherein the encoding of the video data comprises: generating a histogram of occurrence indicating an occurrence frequency of intra prediction modes of reconstructed blocks, wherein the reconstructed blocks include at least one adjacent block and at least one non-adjacent block of a current block; selecting at least one intra prediction mode of the current block based on the occurrence histogram; and predicting the current block according to the at least one intra prediction mode, wherein the generating of the occurrence histogram comprises: deriving an occurrence frequency for intra prediction modes of the reconstructed blocks based on a distance between each of the reconstructed blocks and the current block.

[0010] As described above, by providing a video encoding / decoding method and device according to the present embodiment and a recording medium for storing a bitstream generated by the video encoding method / device, it is possible to improve video encoding efficiency and enhance video quality.

[0011] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.

[0012] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT (QuadTree plus BinaryTree TernaryTree) structure.

[0013] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.

[0014] Figure 4 is an example diagram of the surrounding blocks of the current block.

[0015] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.

[0016] FIG. 6 is a block diagram illustrating in detail a portion of an image decoding device according to one embodiment of the present disclosure.

[0017] FIG. 7 is an exemplary diagram showing a template area according to one embodiment of the present disclosure.

[0018] FIG. 8 is an exemplary diagram showing a template and a reference area of ​​the template according to one embodiment of the present disclosure.

[0019] FIG. 9 is an exemplary diagram illustrating spatially adjacent blocks according to one embodiment of the present disclosure.

[0020] FIG. 10 is an exemplary diagram illustrating spatially non-adjacent blocks according to one embodiment of the present disclosure.

[0021] FIG. 11a and FIG. 11b are exemplary diagrams showing the configuration of a histogram of occurrence (HoC) according to one embodiment of the present disclosure.

[0022] FIG. 12a and FIG. 12b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0023] FIG. 13a and FIG. 13b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0024] FIG. 14a and FIG. 14b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0025] FIG. 15 is a flowchart illustrating a method for an image encoding device to encode a current block according to one embodiment of the present disclosure.

[0026] FIG. 16 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0027] Hereinafter, embodiments of the present invention will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known structures or functions will be omitted if they are deemed to obscure the gist of the present embodiments.

[0028] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.

[0029] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).

[0030] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0031] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.

[0032] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.

[0033] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.

[0034] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).

[0035] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.

[0036] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0037] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.

[0038] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.

[0039] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.

[0040] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).

[0041] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.

[0042] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.

[0043] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.

[0044] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values ​​using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.

[0045] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0046] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.

[0047] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.

[0048] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.

[0049] Various methods can be used to minimize the number of bits required to encode motion information.

[0050] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.

[0051] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.

[0052] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.

[0053] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.

[0054] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.

[0055] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0056] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.

[0057] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.

[0058] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.

[0059] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.

[0060] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.

[0061] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.

[0062] The transformation unit (140) transforms residual signals within a residual block having pixel values ​​in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0063] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.

[0064] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.

[0065] The rearrangement unit (150) can perform rearrangement of coefficient values ​​for quantized residual values.

[0066] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.

[0067] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.

[0068] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about the prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about the intra prediction mode) or inter prediction information (information about the encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of the merge mode, and a reference picture index and a differential motion vector in the case of the AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about quantization parameters and information about a quantization matrix.

[0069] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.

[0070] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.

[0071] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.

[0072] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.

[0073] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0074] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.

[0075] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.

[0076] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).

[0077] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.

[0078] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.

[0079] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.

[0080] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.

[0081] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.

[0082] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.

[0083] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.

[0084] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.

[0085] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.

[0086] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.

[0087] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.

[0088] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values ​​with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.

[0089] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.

[0090] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.

[0091] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.

[0092] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.

[0093] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.

[0094] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.

[0095] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.

[0096] The present embodiment relates to encoding and decoding of images (video) as described above. More specifically, the present invention provides an image encoding / decoding method and device for adaptively generating a histogram of occurrences (HoC) using weights according to an intra prediction direction and weights according to a distance between blocks, and a recording medium for storing a bitstream generated by the image encoding method / device.

[0097] The following embodiments may be performed by an intra prediction unit (122) within a video encoding apparatus. In addition, the following embodiments may be performed by an intra prediction unit (542) within a video decoding apparatus.

[0098] The video encoding device can generate signaling information related to the present embodiment in terms of rate distortion optimization in encoding the current block. The video encoding device can encode the signaling information using the entropy encoding unit (155) and then transmit it to the video decoding device. The video decoding device can decode the signaling information related to the decoding of the current block from the bitstream using the entropy decoding unit (510).

[0099] In the following description, the term "target block" may be used interchangeably with the current block or coding unit (CU). Alternatively, the term "target block" may also refer to a portion of a coding unit.

[0100] Also, a value of a flag being true indicates that the flag is set to 1. Also, a value of a flag being false indicates that the flag is set to 0.

[0101] The decoder side includes all or part of the inverse quantization unit (160), the inverse transform unit (165), the prediction unit (120), the adder (170), the loop filter unit (180), and the memory (190) in the video encoding device illustrated in FIG. 1. Alternatively, the decoder side includes all or part of the inverse quantization unit (520), the inverse transform unit (530), the prediction unit (540), the adder (550), the loop filter unit (560), and the memory (570) in the video decoding device illustrated in FIG. 5. With respect to a series of decoding processes, the decoder side of the video encoding device and the decoder side of the video decoding device perform the same operation. The video encoding device determines information related to the operation of the decoder side and signals the determined information to the video decoding device. The video decoding device can decode the signaled information and operate the decoder side based on the decoded information.

[0102] Hereinafter, the derivation of the intra prediction direction is used interchangeably with the derivation of the directional intra prediction mode. The directional intra prediction mode illustrated in Fig. 3a is simply denoted as the intra prediction mode (IPM) and can be distinguished from the planar and DC modes.

[0103] I. Intra Prediction - OBIC (Occurrence-based Intra Coding) Mode

[0104] OBIC mode derives multiple intra prediction directions based on prediction information of spatially adjacent blocks and non-adjacent blocks in the restoration area around the current block, generates corresponding prediction blocks using one or more derived intra prediction directions, and weights and adds the prediction blocks to generate the final prediction block of the current block.

[0105] In OBIC mode, a candidate list of intra prediction modes is constructed based on prediction information of adjacent and non-adjacent blocks to derive an intra prediction mode, and a histogram of occurrences (HoC) is constructed based on the intra prediction direction and block size of blocks in the candidate list. OBIC mode generates a final prediction block by weighting prediction blocks generated according to a planar mode or block vector (BV)-based prediction block, and up to five directional prediction modes selected in the order of the largest cumulative value in the HoC.

[0106] The OBIC mode can use blocks predicted by an intra prediction mode among adjacent blocks to construct a candidate list. For example, blocks predicted by Decoder-side Intra Mode Derivation (DIMD), Occurrence-Based Intra Coding (OBIC), Spatial Geometry Partitioning Mode (SGPM), Template-based Intra Mode Derivation (TIMD), and blocks predicted by an intra prediction mode can be used. In addition, blocks that do not use an intra prediction direction but derive an intra prediction mode to apply a non-separable second-order transform, such as blocks predicted by Extrapolation filter-based Intra Prediction (EIP), Matrix-weighted Intra Prediction (MIP), Intra Template Matching Prediction (IntraTMP), etc., can also be used. The candidate list can include up to 31 candidate blocks (hereinafter, used interchangeably with candidates). Duplicate candidates are removed during the process of constructing the candidate list. The candidate list can be reordered based on the distance between the current block and the candidate block. Ultimately, the candidate list is constructed using only the 20 closest candidates.

[0107] The OBIC mode constructs a HoC using the intra prediction modes and block sizes of blocks in the candidate list. The histogram size is accumulated for the intra prediction modes of the candidate blocks, and the accumulated value is equal to the size of the candidate block. If the candidate block is generated according to multiple intra prediction modes, such as DIMD, OBIC, SGPM, and TIMD, the histogram size is accumulated for all intra prediction modes by the size of the candidate block. The HoC is constructed by repeatedly performing the above process for all candidate blocks.

[0108] OBIC mode selects up to five prediction modes in descending order of accumulated value in HoC, and determines the weight of each prediction mode according to the accumulated value. The final prediction block is generated by weighting up to five prediction blocks generated according to the selected prediction mode and the prediction blocks generated based on Planar mode or BV. At this time, a prediction block based on Planar mode or BV mode can be selected based on the template cost of the left and upper areas of the restored current block. BV information can be obtained from a block predicted according to IntraTMP or IBC (Intra Block Copy) mode among spatially adjacent or non-adjacent blocks.

[0109] The following embodiments are described with a focus on a video decoding device, but can be implemented in the same or similar manner in a video encoding device. Alternatively, the following embodiments are described with a focus on the decoder side of a video decoding device, but can also be implemented in the same or similar manner in the decoder side of a video encoding device.

[0110] II. Embodiments according to the present disclosure

[0111] FIG. 6 is a block diagram illustrating in detail a portion of an image decoding device according to one embodiment of the present disclosure.

[0112] Hereinafter, “current block”, “current coding block”, “current prediction block”, and “current prediction unit block” have the same meaning and can be used interchangeably.

[0113] The video decoding device according to the present embodiment determines prediction and transformation units, and performs prediction and inverse transformation on the current block corresponding to the determined unit using the determined prediction technique and prediction mode, thereby finally generating a restoration block of the current block. The example illustrated in FIG. 6 may be performed by the inverse transformation unit (530), the prediction unit (540), and the adder (550) of the video decoding device. Meanwhile, the same operations as the example illustrated in FIG. 6 may be performed by the inverse transformation unit (165), the picture division unit (110), the prediction unit (120), and the adder (170) of the video encoding device. At this time, the video decoding device uses encoding information parsed from the bitstream, but the video encoding device may use encoding information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment will be described with reference to the video decoding device.

[0114] As shown in the example of FIG. 5, the prediction unit (540) includes an intra prediction unit (542) and an inter prediction unit (544) depending on the prediction technology, but as shown in FIG. 6, the prediction unit (540) may include all or part of the prediction unit determination unit (602), the prediction technology determination unit (604), the prediction mode determination unit (606), and the prediction execution unit (608).

[0115] The prediction unit determination unit (602) determines a prediction unit (PU). The prediction technique determination unit (604) determines a prediction technique (e.g., intra prediction, inter prediction, IBC (Intra Block Copy) mode, palette mode, a technique that mixes intra and inter prediction, etc.) for the prediction unit. The prediction mode determination unit (606) determines a detailed prediction mode for the prediction technique. The prediction execution unit (608) generates a prediction block of the current block according to the determined prediction mode.

[0116] The inverse transform unit (530) includes all or part of the inverse transform unit determination unit (610), the inverse transform kernel determination unit (612), and the inverse transform execution unit (614). The inverse transform unit determination unit (610) determines a transformation unit for the inverse quantization signals (i.e., inverse quantization transform coefficients) of the current block, and can correct the prediction signal of the transformation unit before performing inverse transform for the determined transformation unit. The inverse transform kernel determination unit (612) determines an inverse transform kernel, and the inverse transform execution unit (614) generates residual samples by inversely transforming the transformation unit expressed by the inverse quantization transform coefficients.

[0117] Hereinafter, the transform unit (TU) on the encoding side can be used interchangeably with the transform block. The inverse transform unit and inverse transform block on the decoding side correspond to the TU and transform block, respectively. Therefore, the inverse transform unit and inverse transform block can be used interchangeably with the TU and transform block.

[0118] A transform unit is a unit that determines whether to perform an inverse transform with respect to transform coefficients and represents a unit in which information about the inverse transform is transmitted. An image decoding device determines a kernel to be applied to a transform unit and performs an inverse transform of transform coefficients based on the determined kernel. At least one kernel may be determined for one transform unit. According to an embodiment, an inverse transform of N orders may be applied, and the sizes of the transform unit and the inverse transform kernel in each order may not be the same. For example, when an inverse transform of N orders is applied to a current transform unit block (hereinafter, transform block or current transform block), the sizes of the kernels used for the inverse transform of each order may not be the same. That is, the image decoding device may apply an inverse transform to some of the input transform coefficients.

[0119] An adder (550) adds a prediction block and residual samples to generate a restoration block. The restoration block is stored in memory and can be used to predict other blocks.

[0120] If the color format of the input video is a YUV format (such as YUV420, YUV411, YUV422, YUV444), the video decoding device can perform prediction and restoration of the chroma component after performing prediction and restoration of the luma component. That is, the luma component and the chroma component can be sequentially restored by the components illustrated in Fig. 6. Meanwhile, if the color format of the input video is RGB, the video encoding device can perform color format conversion from RGB to YUV and then encode the converted video. Here, in the case of the YUV format, the color format represents the correspondence between the pixels of the luma component and the pixels of the chroma component.

[0121] The prediction unit determination unit (602) determines the size and shape of the prediction target block, and may utilize all direct or indirect information transmitted from the video encoding device. The prediction unit determination unit (602) may utilize direct information related to the size and shape of the current block, and may utilize information that may influence the determination of the size and shape of the current block, such as the number of divisions, depth, shape of division, direction of division, size information related to the minimum division block, and division information / prediction mode of the decoded neighboring blocks.

[0122] The prediction unit determined by the prediction unit determination unit (602) may be a current block, one of the sub-blocks into which the current block is divided, a set of pixels, or a single pixel. The prediction unit may include size information and shape information for performing prediction of chroma components and luma components.

[0123] The prediction unit can be determined dependently or independently for the chroma component and the luma component. Dependent determination means that the prediction unit of the luma component or chroma components is not determined for each component, but rather, when the prediction unit of one component is determined, the units of the other components or components are determined with a corresponding size and shape. In this case, one component may correspond to one or more of the luma component and the chroma component. That is, the luma component may be determined based on information of the chroma components. For example, the prediction unit of the chroma component may have a size corresponding to the prediction unit of the luma component depending on the color format. In the case of dependent determination, information about the prediction unit of the other component corresponding to one component, i.e., the dependently determined component, may be omitted. Independent determination means that the prediction units of the luma component and the chroma component are determined separately. In the case of independent determination, information about the prediction unit of each component may be signaled separately.

[0124] The prediction technique determination unit (604) determines a prediction technique for each prediction unit. As described above, the prediction technique may be one of inter prediction, intra prediction, IBC mode, palette mode, or a technique combining intra and inter prediction. In this case, the prediction technique for the chroma component may be determined in the same manner as the prediction technique for the corresponding luma component, without separate signaling or parsing of information. In some embodiments, a technique combining intra and inter prediction may be included in inter prediction.

[0125] For example, if the prediction technology of the current block is not intra prediction, the video decoding device parses 1-bit flag information. For example, if the parsed flag indicates a skip mode, the video decoding device determines the prediction mode of the current block to be a merge mode of inter prediction or an IBC merge mode. In the case of a skip mode, the video decoding device can use the prediction signals as restoration signals without performing an inverse transformation process (i.e., without parsing the residual signals). Here, skip may mean a method of not transmitting motion information (for example, a motion vector, a reference picture, a reference picture list, etc.) or transmitting motion information using only at least one piece of syntax information, and may also mean that the residual signal of the current block is not transmitted.

[0126] On the other hand, if the parsed flag does not indicate a Skip mode for the current block, the prediction technique determination unit (604) can parse a series of 1-bit flags to determine the prediction technique of the current block as one of techniques such as inter prediction, intra prediction, IBC mode, palette mode, etc.

[0127] For example, if Skip is not applied to the current block and the prediction technique is determined to be Inter-Prediction or IBC mode, the video decoding device parses a 1-bit flag. Depending on the parsed flag, the prediction mode of the current block can be determined as either General Merge mode or Advanced Motion Vector Prediction (AMVP) mode.

[0128] The prediction mode determination unit (606) determines the detailed prediction mode of the current prediction unit block in relation to the prediction technology.

[0129] For example, if the prediction technology of the current block is inter prediction, the prediction mode of the current block may be a mode that generates a prediction block of the current block using the parsed motion information of the current block. If the motion information of the current block includes multiple motions, multiple prediction blocks may be generated using motion compensation based on each motion, and the multiple prediction blocks may be weighted and combined to generate a prediction block of the current block.

[0130] As another example, if the prediction technique of the current block is inter prediction, the prediction mode of the current block may be a geometric partitioning-based prediction mode (hereinafter, geometric partitioning mode (GPM)). In GPM, the image decoding device divides the current block into two or more sub-regions according to geometric partitioning, generates prediction blocks according to one or more motion compensations based on the parsed motion information and prediction mode information of the current block, and weights and combines the generated plurality of prediction blocks to generate final prediction signals of the current block. According to an embodiment, if the prediction technique is inter prediction and the geometric partitioning mode is determined, at least one block among the sub-blocks in the current block may be predicted according to inter prediction.

[0131] As another example, if the prediction technique of the current block is inter prediction and intra-predicted samples are used to generate the final predicted block of the current block, the surrounding restored region of the current block may be defined as a template, and an intra-prediction mode may be derived using the template. At this time, the template may also include regions that are not adjacent to the current block. Thereafter, the final predicted block of the current block may be generated using the derived intra-prediction mode. For example, the final predicted block of the current block may be generated by weighting the intra-prediction block determined by derivation and / or parsing and the inter-prediction block determined by derivation and / or parsing. The weights may vary depending on the location of each sample. The image decoding device may adaptively determine the weights using prediction information of the current block, information on the surrounding restored region, the aspect ratio of the current block, etc.

[0132] If the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a mode for generating a prediction block of the current block based on at least one of a directional prediction mode, a planar mode (Horizontal Planar, Vertical Planar, or Regular Planar), a DC mode, an EIP (Extrapolation filter-based intra prediction) prediction mode, a matrix-based prediction mode (for example, MIP), or a prediction mode based on correlation between components (e.g., CCLM (Cross component linear model), MMLM (Multi model CCLM), CCCM (Convolutional cross component model), LBCCP (Local boosting cross component prediction), BVG-CCCM (Block vector guided CCCM), CCLM with slope adjustment, Multi model CCCM, GLCCCM (Gradient and location based CCCM), CCP-merge, GLM (Gradient linear model), DDCCP (Decoder derived cross component prediction), etc.).

[0133] As another example, if the prediction technique of the current block is intra prediction, the prediction mode of the current block may be a spatial geometry partitioning-based prediction mode (SGPM). SGPM divides the current block into one or more sub-regions according to geometric partitioning, generates a prediction block for each region using intra-prediction modes including different directional prediction modes, planar mode, DC mode, etc., and weights and combines each prediction block to generate a prediction block of the current block.

[0134] For example, if the prediction technology of the current block is intra prediction, the prediction mode of the current block may be a matrix-based intra prediction mode (for example, MIP (Matrix-based intra prediction)). The MIP mode may signal / parse and / or derive an index of a matrix based on a predefined matrix according to an agreement between a video encoding device and a video decoding device, or may signal / parse and / or derive a matrix and generate a prediction block of the current block based on the matrix.

[0135] As another example, if the prediction technique for the current block is intra prediction, the prediction mode for the current block may be intra-template matching prediction (IntraTMP). IntraTMP mode defines a restored area surrounding the current block as a template, and performs template matching on the restored area surrounding the current block to generate a predicted block. The template may also include areas not adjacent to the current block.

[0136] As another example, if the prediction technique for the current block is intra prediction, the surrounding restoration region of the current block can be defined as a template, and an intra prediction mode can be derived based on the template. A prediction block for the current block can be generated based on the derived intra prediction mode. The template can include both adjacent and non-adjacent regions to the current block. The non-adjacent region can be an area within a certain pixel line distance from the current block.

[0137] For example, there are various methods for deriving the intra prediction mode of the current block. Among these methods, one or more may be used based on the signaling / parsing of flags and / or indices. As another example, for a specific prediction mode, a fixed method may be used.

[0138] As another example, if the prediction technique of the current block is intra prediction and the surrounding reconstructed region of the current block is used as a template to derive the intra prediction mode, the prediction mode of the current block may be the Decoder-side Intra Mode Derivation (DIMD) mode. The DIMD mode constructs a histogram of gradients (HoG) based on directionality in the template region, derives multiple intra prediction modes from the directionality and magnitude of the directionality of the HoG, and can weight and combine multiple intra prediction blocks generated according to the multiple intra prediction modes to generate a final prediction block. The DIMD mode utilizes the Sobel operation, horizontal / vertical gradient operations, etc. to calculate the directionality and magnitude of the directionality of each pixel in the template region, and constructs the HoG by accumulating the magnitude of the directionality for each direction.

[0139] In DIMD mode, the template area can be one of (a) to (d) in Fig. 7. For example, as in (a) of Fig. 7, an 'L'-shaped area including three pixel lines at the left, top, and top left in the restored adjacent area around the current block can be defined as the template area. If the lower left area is restored based on the current block, the template area can be defined including the lower left area, as in (b) of Fig. 7. If the upper right area is restored, the template area can be defined including the upper right area, as in (c) of Fig. 7. If both the lower left area and the upper right area are restored, the template area can be defined including both the lower left area and the upper right area, as in (d) of Fig. 7.

[0140] The DIMD mode selects at least one directionality from the HoG according to an accumulated value to derive at least one intra prediction mode, and generates at least one prediction block according to the at least one intra prediction mode. The DIMD mode can generate a final prediction block by weighting at least one prediction block and a prediction block based on a non-directional mode. The non-directional mode represents a Planar mode or a BV mode, and the non-directional mode can be determined based on a template cost. BV information for the BV mode can be obtained from a block generated according to the IntraTMP or IBC (Intra Block Copy) mode among spatially adjacent or non-adjacent blocks.

[0141] As another example, if the prediction technique of the current block is intra prediction and the surrounding restored region of the current block is used as a template to derive the intra prediction mode, the prediction mode of the current block may be the TIMD (Template based Intra Mode Derivation) mode. The TIMD mode uses the surrounding restored region of the template region as a reference region of the template, as shown in FIG. 8. The TIMD mode generates intra prediction samples for the template region from the reference region of the template, and selects multiple intra prediction modes in the order of decreasing or increasing cost between the predicted samples for the template region and the restored samples. The TIMD mode can generate a final prediction block by weighting blocks generated according to multiple intra prediction modes and blocks predicted in the DC mode or the Planar mode.

[0142] When intra-predicting a template region based on a reference template region, a directional prediction mode can be selected from a Most Probable Mode (MPM) list. The MPM list can be constructed in the following manner. The MPM list can be constructed using intra-prediction modes of spatially adjacent blocks, multiple intra-prediction modes derived by the DIMD method, and intra-prediction modes of spatially non-adjacent blocks. The MPM list can be sorted according to template costs of the restored left and upper regions of the current block according to the intra-prediction mode. After the MPM list is sorted, intra-prediction modes adjacent to intra-prediction modes existing in the MPM list can be additionally inserted into the list. In addition, intra-prediction modes predefined as a table including a DC mode, a vertical mode, and a horizontal mode can be additionally inserted into the list.

[0143] A final prediction block of the current block can be generated by weighting multiple intra prediction blocks based on the cost difference between the samples of the predicted template region and the samples of the reconstructed template region according to the intra prediction mode selected from the MPM list and the DC mode or the Planar mode. Alternatively, one intra prediction block with the smallest or largest cost can be determined as the final prediction block. The cost between the predicted samples and the reconstructed samples in the template region can be calculated according to various methods. For example, the Sum of Absolute Difference (SAD) can be used. The SAD is generated by summing the absolute differences between the predicted samples and the reconstructed samples in the template region on a pixel-by-pixel basis. Alternatively, the Sum of Absolute Transformed Difference (SATD) can be used. The SATD is generated by performing a Hadamard transform on the differences between the predicted samples and the reconstructed samples in the template region on a pixel-by-pixel basis and summing the absolute values ​​of the transformed values.

[0144] As another example, if the prediction technique of the current block is intra prediction, the final prediction block can be generated by weighting the prediction block generated according to intra template matching and the prediction block generated using the intra prediction mode, Planar, and / or DC mode, or parsed intra prediction mode derived based on the template of the current block.

[0145] The prediction execution unit (608) generates a final prediction block of the current decryption block (hereinafter, used interchangeably with the current block) according to the determined prediction technology and prediction mode.

[0146] As an example, the prediction performing unit (608) performs prediction according to the prediction mode determined as described above to generate a prediction block of the current block, and the adder (550) adds the prediction block of the current block and residual samples (i.e., residual block) to generate a restoration block.

[0147] For example, if the intra prediction mode of the current block is determined to be OBIC (Occurrence-based Intra Coding) mode based on signaling / parsing of a 1-bit flag, the video decoding device can construct a candidate list using spatially adjacent blocks and non-adjacent blocks as shown in FIGS. 9 and 10 . In FIGS. 9 and 10 , the numbered candidate blocks represent one CU, and the positions of each candidate block are expressed relatively. For example, block 4 is located immediately to the right of block 1, block 6 is located immediately to the right of block 4, and block 12 is located immediately above block 6. The assigned numbers indicate the construction order of the candidate list, and the candidate positions for HoC construction and the candidate list construction order may be changed. Among the surrounding blocks, a block generated in the intra prediction mode, a block generated in the inter prediction mode, and / or a block generated in the IBC mode may be searched, and the searched block may be used as a candidate block.

[0148] As an example, a video decoding device can generate a HoC from an intra prediction mode and the size of a candidate block. By applying a process of accumulating the intra prediction mode of a candidate block by the size of the candidate block to all blocks in the candidate list, the video decoding device can construct the HoC.

[0149] For example, if a candidate block is generated according to an inter prediction or IBC mode, the video decoding device can derive an intra prediction mode and form a HoC by accumulating the derived intra prediction mode by the size of the candidate block. As a method of deriving the intra prediction mode, the derived intra prediction mode can be used as is to apply a non-separable second transform, or a new intra prediction mode can be derived based on the HoG in the area of ​​the restored candidate block. The video decoding device performs a Sobel operation on a pixel-by-pixel basis in the area of ​​the restored candidate block to calculate the directionality and magnitude of the directionality of each pixel, and accumulates the magnitude of the directionality for all pixels to form the HoG. The direction with the largest accumulated value in the HoG can be derived as the intra prediction mode of the candidate block. To calculate the directionality of each pixel, a Sobel operation, a difference operation, etc. can be used.

[0150] For example, when generating a HoC from candidate blocks, the video decoding device can construct a histogram by accumulating the sizes of the candidate blocks by assigning weights. If the candidate blocks are blocks generated based on multiple intra prediction modes, such as DIMD, TIMD, OBIC, and SGPM, the video decoding device can assign different accumulated weights to each intra prediction mode, or assign accumulated weights based on the distance between the current block and the candidate blocks.

[0151] For example, if a candidate block is intra-predicted and the prediction mode specifies the intra-prediction direction, the video decoding device may use the candidate block to construct a HoC. The video decoding device may construct the HoC by accumulating the intra-prediction mode of the candidate block by the size of the candidate block.

[0152] As another example, if a candidate block is intra-predicted and the prediction mode is a non-directional intra-prediction mode such as MIP, IntraTMP, or EIP, but an intra-prediction direction is derived to apply a non-separable secondary transform to the candidate block, the video decoding device can use the candidate block to construct a HoC. For the intra-prediction mode derived to apply the non-separable secondary transform, the video decoding device can construct the HoC by accumulating as much as the size of the candidate block. The method of deriving the intra-prediction mode to apply the non-separable secondary transform can be performed based on various embodiments. For example, the video decoding device can construct the HoG by performing a Sobel operation on the prediction samples generated in the non-directional intra-prediction mode on a pixel-by-pixel basis (or calculating gradients for the horizontal and vertical directions) to calculate the directionality and the magnitude of the directionality for each pixel, and accumulating the magnitude of the directionality for all pixels. The prediction mode with the largest accumulated value in the HoG can be derived as the intra prediction mode for the non-separable second-order transform. Sobel operations, difference operations, etc. can be used to estimate the directionality of each pixel.

[0153] As another example, if a candidate block is generated according to an inter prediction or IBC mode and an intra prediction direction is derived to apply a non-separable secondary transform, the video decoding device can use the candidate block to construct a HoC. For the intra prediction mode derived to apply the non-separable secondary transform, the video decoding device can construct the HoC by accumulating as much as the size of the candidate block. A method for deriving an intra prediction mode to apply the non-separable secondary transform can be performed based on various embodiments. For example, the video decoding device can perform a Sobel operation on a pixel-by-pixel basis on prediction samples generated in the inter prediction or IBC mode (or calculate gradients for horizontal and vertical directions) to calculate the directionality and the magnitude of the directionality for each pixel, and construct the HoG by accumulating the magnitude of the directionality for all pixels. The prediction mode with the largest accumulated value in the HoG can be derived as the intra prediction mode for the non-separable secondary transform. To calculate the directionality of each pixel, Sobel operation, difference operation, etc. can be used.

[0154] As another example, if the candidate block is an inter-predicted block, the video decoding device can derive an intra prediction direction and use the candidate block for constructing the HoC. For example, the video decoding device can perform a Sobel operation on the restored candidate block pixel by pixel (or compute gradients for the horizontal and vertical directions) to calculate the directionality and magnitude of the directionality for each pixel, and accumulate the magnitude of the directionality for all pixels to construct the HoG. A Sobel operation, a difference operation, etc. can be used to calculate the directionality of each pixel. The video decoding device can derive the prediction mode with the largest accumulated value in the HoG as the intra prediction mode of the candidate block, and construct the HoC by accumulating the derived intra prediction mode by the size of the candidate block.

[0155] As another example, if the candidate block is a block predicted in the IBC mode, the video decoding device can derive an intra prediction direction and use the candidate block for constructing the HoC. For example, the video decoding device can perform a Sobel operation on the restored candidate block pixel by pixel (or compute gradients for the horizontal and vertical directions) to calculate the directionality and magnitude of the directionality for each pixel, and can construct the HoG by accumulating the magnitude of the directionality for all pixels. A Sobel operation, a difference operation, etc. can be used to calculate the directionality of each pixel. The video decoding device can derive the prediction mode with the largest accumulated value in the HoG as the intra prediction mode of the candidate block, and construct the HoC by accumulating the derived intra prediction mode by the size of the candidate block.

[0156] FIG. 11a and FIG. 11b are exemplary diagrams showing the configuration of a HoC according to one embodiment of the present disclosure.

[0157] As shown in Fig. 11a, if a candidate block is predicted according to a prediction mode that generates multiple prediction blocks according to multiple intra prediction modes, such as DIMD, OBIC, and TIMD modes, and weights and combines the multiple prediction blocks to generate a final prediction block, the video decoding device can use the candidate block to construct a HoC. As shown in Fig. 11b, when constructing a HoC for multiple intra prediction modes, the video decoding device can construct the HoC by accumulating a value obtained by multiplying the size of the candidate block by a weight for each intra prediction direction. For example, if a candidate block is predicted according to a mode that derives multiple intra prediction modes (Intra Prediction Modes (IPMs)) based on the size of the histogram, such as DIMD mode or OBIC mode, the video decoding device can determine the size of the histogram for each intra prediction mode as a weight for each intra prediction mode. For example, the weights for each intra prediction mode may be determined such that a larger weight is assigned to an intra prediction direction with a larger histogram size. If a candidate block is predicted according to a mode that derives multiple intra prediction modes based on template costs, such as the TIMD mode, the video decoding device may determine the weights for each intra prediction mode based on the template costs for each intra prediction mode. For example, the weights for each intra prediction direction may be determined such that a larger weight is assigned to an intra prediction mode with a lower cost.

[0158] The HoC of the current block can be constructed as in Equation 1 using the weights for each intra prediction mode associated with the candidate block.

[0159]

[0160] In Equation 1, numSamples represents the size of the candidate block, and i is an index indicating one of the multiple intra prediction modes of the candidate block. IPMWeight[i] is the weight of the corresponding intra prediction mode and corresponds to w[i] in FIGS. 11a and 11b.

[0161] FIG. 12a and FIG. 12b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0162] As shown in Fig. 12a, when a candidate block is divided into multiple regions, such as SGPM, and each region is predicted according to a different intra prediction mode, the video decoding device can use the candidate block to construct a HoC. As shown in Fig. 12b, when constructing a HoC for the intra prediction mode of each region, the video decoding device can construct the HoC by accumulating the size of each region. The video decoding device can construct the HoC by utilizing all of the regions, and some of the regions among the regions can be used to construct the HoC depending on the size difference or ratio between the regions. As shown in Equation 2, the HoC can be adaptively constructed using the size of the regions.

[0163]

[0164] In mathematical expression 2, Region i can represent, for example, Region 0 or Region 1 in FIG. 12a.

[0165] As another example, if a candidate block is divided into multiple regions, such as in GPM mode, and each region is predicted according to different inter prediction, the video decoding device can use the candidate block to construct the HoC. The video decoding device can perform a Sobel operation on each pixel of each restored region to calculate the directionality and the magnitude of the directionality, and can construct the HoG of each region by accumulating the magnitude of the directionality for the pixels in each region. The video decoding device can derive the prediction mode with the largest accumulated value in each HoG as the intra prediction mode of each region, and can construct the HoC by accumulating the intra prediction modes derived from each region by the size of each region. The video decoding device can construct the HoC by utilizing all of the regions, and some of the regions among the regions can be used to construct the HoC depending on the size difference or ratio between the regions.

[0166] For example, when constructing a HoC, the video decoding device can adaptively consider the distance between a candidate block and a current block. For example, a HoC can be constructed with different weights assigned to adjacent blocks, such as in FIG. 9, and non-adjacent blocks, such as in FIG. 10. The candidate blocks can be determined by searching surrounding blocks, and the surrounding blocks can include adjacent blocks and non-adjacent blocks. Different weights can be assigned to each candidate block depending on the distance between the candidate blocks and the current block. For example, the greater the distance between the candidate block and the current block, the smaller the weight assigned to the candidate block. As another example, the candidate blocks can be divided into multiple groups depending on the distance between the candidate blocks and the current block, and different weights can be assigned to each group. For example, the candidate blocks can be divided into three groups depending on the distance between the candidate blocks and the current block, and each group can be assigned a weight of {4, 2, 1}.

[0167] Below, the distance-based weights can be used interchangeably with the distance factor.

[0168] FIG. 13a and FIG. 13b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0169] When configuring a HoC, the video decoding device may consider the distance between the candidate block and the current block, as shown in FIG. 13a. The video decoding device may calculate the total distance for all candidate blocks by adding up all distances between the current block and the candidate blocks, and normalize the distance between each candidate block and the current block based on the total distance. The amplitude accumulated in the HoC may be set to be inversely proportional to the normalized distance. As shown in FIG. 13b, the HoC size for the intra prediction mode of candidate block 10 may be determined as (total distance × numSamples

[0010] ) / dist

[0010] . In FIG. 13b, "total dist" represents the total distance, and total dist / dist

[0010] represents a weight according to the distance, i.e., a distance factor.

[0170] As an example, the distance can be calculated based on the upper left sample, center sample, etc. of the current block and the candidate block. To calculate the distance, the L-1 distance, L-2 distance, L-∞ distance, etc. can be used.

[0171] FIG. 14a and FIG. 14b are exemplary diagrams showing the configuration of HoC according to another embodiment of the present disclosure.

[0172] For example, if a candidate block is predicted according to a prediction mode that generates prediction blocks according to multiple intra prediction modes, such as DIMD, OBIC, TIMD, and SGPM modes, and weights and combines the multiple prediction blocks to generate a final prediction block, the image decoding device can configure a HoC by considering the distance between the candidate block and the current block and the weight according to the intra prediction mode. As shown in FIG. 14a, if the neighboring block 10 is predicted with SGPM, the size of the HoC for the intra prediction mode of partition region 0 can be determined as (total distance × numSamples of Region 0) / dist

[0010] , and the size of the HoC for the intra prediction mode of partition region 1 can be determined as (total distance × numSamples of Region 1) / dist

[0010] . In Fig. 14b, “total dist” represents the total distance, and total dist / dist

[0010] represents the weight according to the distance, i.e., the distance factor.

[0173] As an example, a video decoding device can select a plurality of prediction modes in order of increasing cumulative values ​​in the HoC, and determine a weight of each prediction mode according to the cumulative values. The video decoding device can generate a plurality of prediction blocks according to the selected prediction mode, and generate a final prediction block by weighting the plurality of prediction blocks and a prediction block based on a non-directional mode. The non-directional mode represents a Planar mode or a BV mode, and can be determined based on a template cost. BV information for the BV mode can be obtained from a block generated according to the IntraTMP or IBC (Intra Block Copy) mode among spatially adjacent or non-adjacent blocks.

[0174] Below, the operations related to inverse transformation and restoration are described.

[0175] In relation to the inverse transformation, the entropy decoding unit (510) decodes the transform coefficients. If a secondary transformation is applied, the entropy decoding unit (510) decodes the quantized secondary transform coefficients. If a secondary transformation is not applied, the entropy decoding unit (510) decodes the quantized primary transform coefficients. The entropy decoding unit (510) parses information such as the quantization method and quantization parameter information.

[0176] The inverse quantization unit (520) inversely quantizes the decoded quantized transform coefficients based on information such as the quantization method and quantization parameter information to generate inverse quantized transform coefficients.

[0177] The inverse transform unit (530) generates residual samples by inversely transforming the TU expressed as inverse quantization transform coefficients. As shown in Fig. 6, the inverse transform unit (530) may include an inverse transform unit determination unit (610), an inverse transform kernel determination unit (612), and an inverse transform execution unit (614).

[0178] The inverse transformation unit determination unit (610) can determine a single TU or a sub-block obtained by dividing a single TU into multiple sub-blocks as the target of transformation. For example, the TU may be the entire current block, which is the target of prediction, or a portion of the current block.

[0179] The inverse transform kernel determination unit (612) can determine a separable vertical and horizontal first-order inverse transform kernel, and / or a non-separable second-order inverse transform kernel, or can determine a non-separable first-order inverse transform kernel.

[0180] The inverse transform performing unit (614) can inversely transform the inverse quantized transform coefficients using the inverse transform kernel determined by the inverse transform kernel determining unit (612). The inverse transform performing unit (614) can perform non-separable first-order inverse transform, non-separable second-order inverse transform, first-order inverse transform, etc.

[0181] As an example, whether to perform a non-separable first-order inverse transform and whether to perform a non-separable second-order inverse transform may be determined by signaling / parsing, or may be determined implicitly based on the size of the current transform block.

[0182] Below, using the cities of FIGS. 15 and 16, a method for adaptively predicting the current block based on occurrence frequency is described.

[0183] FIG. 15 is a flowchart illustrating a method for an image encoding device to encode a current block according to one embodiment of the present disclosure.

[0184] The video encoding device determines a flag indicating the derivation of an intra prediction mode based on a generation histogram (S1500). For example, the video encoding device may determine the flag in terms of rate distortion optimization. Alternatively, the video encoding device may obtain the flag from a higher level.

[0185] The video encoding device can encode a flag.

[0186] The video encoding device checks the flag (S1502).

[0187] If the flag does not instruct the derivation of an intra prediction mode based on an occurrence histogram (No in S1502), the video encoding device can generate a prediction block of the current block using an intra prediction technique other than the derivation of an intra prediction mode based on an occurrence histogram.

[0188] On the other hand, if the flag indicates the derivation of an intra prediction mode based on the occurrence histogram (Yes in S1502), the video encoding device can perform the following steps.

[0189] The video encoding device constructs a candidate list including candidate blocks based on prediction information of neighboring blocks of the current block (S1504). Here, the candidate blocks are searched for in a predetermined order from a predetermined number of neighboring blocks, and the neighboring blocks include restored neighboring blocks and restored non-adjacent blocks. That is, the candidate blocks are restored blocks, and the restored blocks may include at least one adjacent block and at least one non-adjacent block of the current block.

[0190] As an example, the candidate blocks may include, among the surrounding blocks, blocks generated according to intra prediction, blocks generated according to inter prediction, and / or blocks generated according to IBC mode.

[0191] As an example, if each candidate block is predicted with a directional intra prediction mode, the video encoding device can add the intra prediction mode to the candidate list.

[0192] As another example, if each candidate block is generated according to inter prediction or IBC mode, and a directional intra prediction mode is derived according to a gradient operation using prediction samples of each candidate block to apply a non-separable second transform to each candidate block, the video encoding device may use the derived directional intra prediction mode as an intra prediction mode associated with each candidate block. Alternatively, the video encoding device may derive at least one intra prediction mode associated with the candidate block based on a gradient operation using reconstructed samples of the candidate block.

[0193] As another example, if each candidate block is generated according to a non-directional intra prediction mode, but a directional intra prediction mode is derived for applying a non-separable second transform to each candidate block, the video encoding device can use the derived directional intra prediction mode as the intra prediction mode associated with each candidate block. In this case, the directional intra prediction mode can be derived by a gradient operation using prediction samples of a restored block predicted according to the non-directional intra prediction mode.

[0194] The video encoding device may rearrange the candidate list. For example, the video encoding device may rearrange the candidate list based on the distance between the current block and the candidate blocks.

[0195] The video encoding device constructs a generation histogram based on at least one intra prediction mode associated with each candidate block in the candidate list, at least one weight corresponding to the at least one intra prediction mode, a distance between each candidate block and a current block, and all or part of the size of each candidate block (S1506).

[0196] As an example, with respect to at least one intra prediction mode associated with each candidate block, the video encoding device can set the occurrence frequency of each intra prediction mode to the size of each candidate block.

[0197] As another example, with respect to at least one intra prediction mode associated with each candidate block, the video encoding device may set the occurrence frequency of each intra prediction mode as the product of the size of each candidate block and a distance factor. The distance factor may be determined as a value inversely proportional to the distance between each candidate block and the current block.

[0198] As another example, if each candidate block is predicted according to a weighted sum mode that weights prediction blocks generated using multiple intra prediction modes, the video encoding device can set the occurrence frequency of the multiple intra prediction modes as the product of the size of each candidate block and the weight corresponding to each intra prediction mode. For example, the weighted sum mode can be a method that uses multiple intra prediction modes, such as DIMD, TIMD, OBIC, and SGPM.

[0199] As another example, if each candidate block is predicted according to a weighted sum mode that weights and sums prediction blocks generated using a plurality of intra prediction modes, the video encoding device can set the occurrence frequency of the plurality of intra prediction modes as the product of the size of each candidate block, a weight corresponding to each intra prediction mode, and a distance factor. For example, the weighted sum mode can be a method that uses a plurality of intra prediction modes, such as DIMD, TIMD, OBIC, and SGPM. In this case, the distance factor can be determined as a value inversely proportional to the distance between each candidate block and the current block.

[0200] The video encoding device selects at least one intra prediction mode of the current block based on the occurrence histogram (S1508).

[0201] The video encoding device can select a preset number of candidate prediction modes having the largest intensity of occurrence histograms as at least one intra prediction mode.

[0202] The video encoding device generates at least one prediction block of the current block according to at least one intra prediction mode (S1510).

[0203] The video encoding device can calculate at least one weight corresponding to at least one prediction block.

[0204] An image encoding device can induce a non-directional prediction mode, generate a second prediction block of a current block according to the non-directional prediction mode, and generate a second weight corresponding to the second prediction block. One of planar and block vector-based prediction can be used as the non-directional mode.

[0205] The video encoding device can calculate the weight of each prediction block based on the intensity value of the occurrence histogram. For non-directional modes, fixed weights can be applied.

[0206] The video encoding device can generate a final prediction block of the current block by weighting at least one prediction block and a second prediction block using at least one weight and a second weight.

[0207] Thereafter, the video encoding device can generate a residual block by subtracting the prediction block (or the final prediction block) from the current block. The video encoding device can apply transformation / quantization to the residual block to generate transformation coefficients and encode the generated transformation coefficients.

[0208] FIG. 16 is a flowchart illustrating a method for an image decoding device to restore a current block according to one embodiment of the present disclosure.

[0209] The video decoding device obtains a flag that instructs the derivation of an intra prediction mode based on a generation histogram (S1600). For example, the video decoding device can decode the flag from a bitstream.

[0210] The video decoding device checks the flag (S1602).

[0211] If the flag does not instruct the derivation of an intra prediction mode based on an occurrence histogram (No in S1602), the image decoding device can generate a prediction block of the current block using an intra prediction technique other than the derivation of an intra prediction mode based on an occurrence histogram.

[0212] On the other hand, if the flag indicates the derivation of an intra prediction mode based on an occurrence histogram (Yes in S1602), the image decoding device may perform the following steps.

[0213] The video decoding device constructs a candidate list including candidate blocks based on prediction information of neighboring blocks of the current block (S1604). Here, the candidate blocks are searched from a preset number of neighboring blocks in a preset order, and the neighboring blocks include restored neighboring blocks and restored non-adjacent blocks. In other words, the candidate blocks are restored blocks, and the restored blocks may include at least one adjacent block and at least one non-adjacent block of the current block.

[0214] As an example, the candidate blocks may include, among the surrounding blocks, blocks generated according to intra prediction, blocks generated according to inter prediction, and / or blocks generated according to IBC mode.

[0215] As an example, if each candidate block is predicted with a directional intra prediction mode, the video decoding device can add the intra prediction mode to the candidate list.

[0216] As another example, if each candidate block is generated according to inter prediction or IBC mode, and a directional intra prediction mode is derived by a gradient operation using prediction samples of each candidate block to apply a non-separable second transform to each candidate block, the image decoding device may use the derived directional intra prediction mode as an intra prediction mode associated with each candidate block. Alternatively, the image decoding device may derive at least one intra prediction mode associated with the candidate block based on a gradient operation using reconstructed samples of the candidate block.

[0217] As another example, if each candidate block is generated according to a non-directional intra prediction mode, but a directional intra prediction mode is derived for applying a non-separable second-order transform to each candidate block, the image decoding device can use the derived directional intra prediction mode as the intra prediction mode associated with each candidate block. In this case, the directional intra prediction mode can be derived by a gradient operation using prediction samples of a restored block predicted according to the non-directional intra prediction mode.

[0218] The video decoding device can rearrange the candidate list. For example, the video decoding device can rearrange the candidate list based on the distance between the current block and the candidate blocks.

[0219] The video decoding device constructs a generation histogram based on at least one intra prediction mode associated with each candidate block in the candidate list, at least one weight corresponding to the at least one intra prediction mode, a distance between each candidate block and the current block, and all or part of the size of each candidate block (S1608).

[0220] As an example, with respect to at least one intra prediction mode associated with each candidate block, the video decoding device can set the occurrence frequency of each intra prediction mode to the size of each candidate block.

[0221] As another example, with respect to at least one intra prediction mode associated with each candidate block, the video decoding device may set the occurrence frequency of each intra prediction mode as the product of the size of each candidate block and a distance factor. The distance factor may be determined as a value inversely proportional to the distance between each candidate block and the current block.

[0222] As another example, if each candidate block is predicted according to a weighted sum mode that weights prediction blocks generated using multiple intra prediction modes, the video decoding device can set the occurrence frequency of the multiple intra prediction modes as the product of the size of each candidate block and the weight corresponding to each intra prediction mode. For example, the weighted sum mode can be a method that uses multiple intra prediction modes, such as DIMD, TIMD, OBIC, and SGPM.

[0223] As another example, if each candidate block is predicted according to a weighted sum mode that weights and sums prediction blocks generated using a plurality of intra prediction modes, the image decoding device can set the occurrence frequency of the plurality of intra prediction modes as the product of the size of each candidate block, a weight corresponding to each intra prediction mode, and a distance factor. For example, the weighted sum mode can be a method that uses a plurality of intra prediction modes, such as DIMD, TIMD, OBIC, and SGPM. In this case, the distance factor can be determined as a value inversely proportional to the distance between each candidate block and the current block.

[0224] The video decoding device selects at least one intra prediction mode of the current block based on the occurrence histogram (S1610).

[0225] The video decoding device can select a preset number of candidate prediction modes having the largest intensity of occurrence histograms as at least one intra prediction mode.

[0226] The video decoding device generates at least one prediction block of the current block according to at least one intra prediction mode (S1612).

[0227] The video decoding device can calculate at least one weight corresponding to at least one prediction block.

[0228] The video decoding device can induce a non-directional prediction mode, generate a second prediction block of the current block according to the non-directional prediction mode, and generate a second weight corresponding to the second prediction block. One of planar and block vector-based prediction can be used as the non-directional mode.

[0229] The video decoding device can calculate the weights of each prediction block based on the intensity values ​​of the occurrence histogram. For non-directional modes, fixed weights can be applied.

[0230] The video decoding device can generate a final prediction block of the current block by weighting at least one prediction block and a second prediction block using at least one weight and a second weight.

[0231] An image decoding device can decode quantized transform coefficients of a current block from a bitstream and apply inverse quantization / inverse transformation to the quantized transform coefficients to generate a residual block. The image decoding device can reconstruct the current block by adding the prediction block (or final prediction block) and the residual block.

[0232] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of ​​one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.

[0233] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.

[0234] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).

[0235] The above description is merely an example of the technical idea of ​​the present embodiment, and those skilled in the art to which the present embodiment pertains may make various modifications and variations without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of ​​the present embodiment, but to explain it, and the scope of the technical idea of ​​the present embodiment is not limited by these embodiments. The protection scope of the present embodiment should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.

[0236]

[0237] CROSS-REFERENCE TO RELATED APPLICATION

[0238] This patent application claims priority to Korean patent application No. 10-2024-0120510, filed on September 5, 2024, the entire contents of which are incorporated herein by reference.

Claims

1. In a method for restoring a current block performed by an image decryption device, A step of generating a histogram of occurrence representing the frequency of occurrence of intra prediction modes of restored blocks, wherein the restored blocks include at least one adjacent block and at least one non-adjacent block of the current block; A step of selecting at least one intra prediction mode of the current block based on the occurrence histogram; and A step of predicting the current block according to at least one intra prediction mode. Including, The steps for generating the above occurrence histogram are: A method comprising a step of deriving an occurrence frequency for intra prediction modes of the restored blocks based on the distance between each of the restored blocks and the current block.

2. In paragraph 1, Further comprising a step of decoding a flag indicating derivation of an intra prediction mode based on the above occurrence histogram, A method comprising a step of selecting at least one intra prediction mode of the current block when the flag indicates derivation of an intra prediction mode based on the occurrence histogram.

3. In paragraph 1, In the case where there is a predicted restoration block according to a weighted sum mode that weights and sums prediction blocks generated using multiple intra prediction modes among the above restoration blocks, the step of generating the occurrence histogram is as follows: A method comprising a step of deriving an occurrence frequency for each of the plurality of intra prediction modes based on weights corresponding to each of the plurality of intra prediction modes of the above restoration block.

4. In paragraph 3, The above weighted sum mode is DIMD (Decoder-side Intra Mode Derivation) or TIMD (Template-based Intra Mode Derivation).

5. In paragraph 1, The step of predicting the current block above is: A step of generating at least one prediction block using at least one intra prediction mode of the current block; A step of calculating at least one weight corresponding to the at least one intra prediction mode based on the occurrence frequency of the at least one intra prediction mode; and A step of generating a prediction block of the current block by applying at least one of the weights to at least one prediction block. A method comprising:

6. In paragraph 1, The steps for generating the above occurrence histogram are: A method comprising a step of deriving an intra prediction mode for a restored block among the restored blocks that is not predicted by the intra prediction mode based on a gradient operation using restored samples or predicted samples of the restored block.

7. In paragraph 1, The steps for generating the above occurrence histogram are: A method further comprising the step of using the derived directional intra prediction mode as an intra prediction mode related to the restored block, when a restored block predicted by an intra prediction mode exists among the restored blocks and a directional intra prediction mode for applying a secondary transformation to the restored block is derived.

8. In paragraph 7. The above derived directional intra prediction mode is, A method characterized in that it is derived by a gradient operation using samples within the predicted block of the restoration block predicted according to the non-directional intra prediction mode.

9. In paragraph 3, The step of deriving the occurrence frequency for each of the above multiple intra prediction modes is: A method for deriving the occurrence frequency based on weights corresponding to each of the plurality of intra prediction modes of the above restoration block and a distance factor, wherein the distance factor is determined as a value inversely proportional to the distance between the above restoration block and the current block.

10. In a method for encoding a current block performed by a video encoding device, A step of generating a histogram of occurrence representing the frequency of occurrence of intra prediction modes of restored blocks, wherein the restored blocks include at least one adjacent block and at least one non-adjacent block of the current block; A step of selecting at least one intra prediction mode of the current block based on the occurrence histogram; and A step of predicting the current block according to at least one intra prediction mode. Including, The steps for generating the above occurrence histogram are: A method comprising a step of deriving an occurrence frequency for intra prediction modes of the restored blocks based on the distance between each of the restored blocks and the current block.

11. In paragraph 10, A step of determining a flag indicating derivation of an intra prediction mode based on the above occurrence histogram; and Step of encoding the above flag Including more, A method comprising a step of selecting at least one intra prediction mode of the current block when the flag indicates derivation of an intra prediction mode based on the occurrence histogram.

12. In paragraph 10, In the case where there is a predicted restoration block according to a weighted sum mode that weights and sums prediction blocks generated using multiple intra prediction modes among the above restoration blocks, the step of generating the occurrence histogram is as follows: A method comprising a step of deriving an occurrence frequency for each of the plurality of intra prediction modes based on weights corresponding to each of the plurality of intra prediction modes of the above restoration block.

13. In paragraph 12, The above weighted sum mode is DIMD (Decoder-side Intra Mode Derivation) or TIMD (Template-based Intra Mode Derivation).

14. In paragraph 12, The step of deriving the occurrence frequency for each of the above multiple intra prediction modes is: A method for deriving the occurrence frequency based on weights corresponding to each of the plurality of intra prediction modes of the above restoration block and a distance factor, wherein the distance factor is determined as a value inversely proportional to the distance between the above restoration block and the current block.

15. In a method for providing video data to a video decoding device, A step of encoding the above video data into a bitstream; and A step of transmitting the above bitstream to the image decoding device Including, The step of encoding the above video data is: A step of generating a histogram of occurrence representing the frequency of occurrence of intra prediction modes of restored blocks, wherein the restored blocks include at least one adjacent block and at least one non-adjacent block of the current block; A step of selecting at least one intra prediction mode of the current block based on the occurrence histogram; and A step of predicting the current block according to at least one intra prediction mode. Including, The steps for generating the above occurrence histogram are: A method comprising a step of deriving an occurrence frequency for intra prediction modes of the restored blocks based on the distance between each of the restored blocks and the current block.

Images