Video coding method and device, and recording medium on which bitstream is stored

Abstract

An image decoding method and device according to the present disclosure can generate an occurrence histogram of the current block on the basis of a reference block of the current block, derive one or more intra prediction modes for the current block on the basis of the occurrence histogram, and generate a prediction block of the current block on the basis of the derived intra prediction mode. Here, the occurrence histogram can include an intra prediction mode and an amplitude value of the reference block.

Description

Video coding method and device, and recording medium storing bitstream

[0001] The present invention relates to a video signal processing method and device.

[0002] The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO / IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC). They completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and have been actively conducting research and development on next-generation compression standards.

[0003] Video compression largely consists of intraprediction, interprediction, transform, quantization, entropy coding, and in-loop filtering. Among these, intraprediction refers to a technique that generates a prediction block for the current block using reconstructed pixels surrounding the current block. The encoder encodes the intraprediction mode used for intraprediction, and the decoder performs intraprediction by reconstructing the encoded intraprediction mode.

[0004] The present disclosure provides a method and device for deriving information regarding intra prediction of a current block.

[0005] The present disclosure provides a method and device for constructing an appearance histogram for deriving information regarding intra prediction of a current block.

[0006] The present disclosure provides a method and device for correcting an appearance histogram of a current block based on a distance between the current block and a reference block.

[0007] The present disclosure provides a method and device for generating a prediction block of a current block through an intra prediction mode derived based on an appearance histogram of the current block.

[0008] The video decoding method and device according to the present disclosure can generate an appearance histogram of the current block based on a reference block of the current block, derive one or more intra prediction modes for the current block based on the appearance histogram, and generate a prediction block of the current block based on the derived intra prediction modes. Here, the appearance histogram can be composed of an intra prediction mode and an amplitude value of the reference block.

[0009] In the image decoding method and device according to the present disclosure, when the reference block has a plurality of intra prediction modes, the appearance histogram may include an amplitude value corresponding to at least one of the plurality of intra prediction modes.

[0010] In the image decoding method and device according to the present disclosure, the amplitude value can be determined based on the size of the reference block.

[0011] In the image decoding method and device according to the present disclosure, the determined amplitude value can be corrected based on the distance between the current block and the reference block. Here, the distance can be calculated based on a reference point of the current block and the reference block, and the reference point can include at least one of the samples located at the center, upper left, upper right, lower left, or lower right within the current block and the reference block.

[0012] In the image decoding method and device according to the present disclosure, the distance can be calculated by weighting distances between a plurality of reference point pairs set based on a plurality of reference points of the current block and a plurality of reference points of the reference block.

[0013] In the image decoding method and device according to the present disclosure, first and second prediction blocks can be generated based on the derived intra prediction mode and a predetermined mode, respectively, and the prediction block of the current block can be generated through a weighted sum between the first prediction block and the second prediction block.

[0014] In the image decoding method and device according to the present disclosure, the predetermined mode may include at least one of a planar mode or a block vector-based mode.

[0015] In the video decoding method and device according to the present disclosure, the weight for the weighted sum may include at least one of a first weight for the first prediction block or a second weight for the second prediction block. Here, the first weight may be derived based on the amplitude value, and the second weight may be a fixed value.

[0016] In the image decoding method and device according to the present disclosure, when an amplitude value of a mode corresponding to an intra prediction mode index 0 exists in the appearance histogram, a weight for the weighted sum can be calculated based on the amplitude value of the derived intra prediction mode and the amplitude value of the mode corresponding to the intra prediction mode index 0.

[0017] In the video decoding method and device according to the present disclosure, when the derived intra prediction mode includes a mode corresponding to an intra prediction mode index of 0, the prediction block of the current block can be generated through a weighted sum between prediction blocks generated based on each of the derived intra prediction modes.

[0018] The video encoding method and device according to the present disclosure can generate an appearance histogram of the current block based on a reference block of the current block, derive one or more intra prediction modes for the current block based on the appearance histogram, and generate a prediction block of the current block based on the derived intra prediction modes. Here, the appearance histogram can be composed of an intra prediction mode and an amplitude value of the reference block.

[0019] A computer-readable recording medium according to the present disclosure can store a bitstream generated by the image encoding method.

[0020] The bitstream transmission method and device according to the present disclosure can transmit a bitstream generated by the image encoding method.

[0021] According to the present disclosure, by generating an appearance histogram of the current block based on a reference block of the current block, more accurate prediction is possible, thereby improving the compression performance of the encoder / decoder.

[0022] According to the present disclosure, by correcting the amplitude value of the appearance histogram based on the distance between the current block and the reference block, more accurate prediction is possible, thereby improving the compression performance of the encoder / decoder.

[0023] According to the present disclosure, by weighting prediction blocks generated based on an intra prediction mode derived from an appearance histogram of a current block, more accurate prediction is possible, thereby improving the compression performance of a decoder / encoder.

[0024] According to the present disclosure, by deriving weights based on the appearance histogram of the current block, efficient prediction becomes possible, and thus the compression performance of the encoder / decoder can be improved.

[0025] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0026] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0027] FIG. 3 illustrates a method for generating a prediction block of a current block according to the present disclosure.

[0028] FIG. 4 is a diagram showing an example of a method for constructing an appearance histogram of a current block according to the present disclosure.

[0029] FIG. 5 is a diagram showing another example of a method for constructing an appearance histogram of a current block according to the present disclosure.

[0030] FIG. 6 is a diagram showing examples of reference blocks for configuring an appearance histogram of a current block according to the present disclosure.

[0031] FIG. 7 is a diagram illustrating an example of a method for generating a prediction block of a current block according to the present disclosure.

[0032] The video decoding method and device according to the present disclosure can generate an appearance histogram of the current block based on a reference block of the current block, derive one or more intra prediction modes for the current block based on the appearance histogram, and generate a prediction block of the current block based on the derived intra prediction modes. Here, the appearance histogram can be composed of an intra prediction mode and an amplitude value of the reference block.

[0033] In the image decoding method and device according to the present disclosure, when the reference block has a plurality of intra prediction modes, the appearance histogram may include an amplitude value corresponding to at least one of the plurality of intra prediction modes.

[0034] In the image decoding method and device according to the present disclosure, the amplitude value can be determined based on the size of the reference block.

[0035] In the image decoding method and device according to the present disclosure, the determined amplitude value can be corrected based on the distance between the current block and the reference block. Here, the distance can be calculated based on a reference point of the current block and the reference block, and the reference point can include at least one of the samples located at the center, upper left, upper right, lower left, or lower right within the current block and the reference block.

[0036] In the image decoding method and device according to the present disclosure, the distance can be calculated by weighting distances between a plurality of reference point pairs set based on a plurality of reference points of the current block and a plurality of reference points of the reference block.

[0037] In the image decoding method and device according to the present disclosure, first and second prediction blocks can be generated based on the derived intra prediction mode and a predetermined mode, respectively, and the prediction block of the current block can be generated through a weighted sum between the first prediction block and the second prediction block.

[0038] In the image decoding method and device according to the present disclosure, the predetermined mode may include at least one of a planar mode or a block vector-based mode.

[0039] In the video decoding method and device according to the present disclosure, the weight for the weighted sum may include at least one of a first weight for the first prediction block or a second weight for the second prediction block. Here, the first weight may be derived based on the amplitude value, and the second weight may be a fixed value.

[0040] In the image decoding method and device according to the present disclosure, when an amplitude value of a mode corresponding to an intra prediction mode index 0 exists in the appearance histogram, a weight for the weighted sum can be calculated based on the amplitude value of the derived intra prediction mode and the amplitude value of the mode corresponding to the intra prediction mode index 0.

[0041] In the video decoding method and device according to the present disclosure, when the derived intra prediction mode includes a mode corresponding to an intra prediction mode index of 0, the prediction block of the current block can be generated through a weighted sum between prediction blocks generated based on each of the derived intra prediction modes.

[0042] The video encoding method and device according to the present disclosure can generate an appearance histogram of the current block based on a reference block of the current block, derive one or more intra prediction modes for the current block based on the appearance histogram, and generate a prediction block of the current block based on the derived intra prediction modes. Here, the appearance histogram can be composed of an intra prediction mode and an amplitude value of the reference block.

[0043] A computer-readable recording medium according to the present disclosure can store a bitstream generated by the image encoding method.

[0044] The bitstream transmission method and device according to the present disclosure can transmit a bitstream generated by the image encoding method.

[0045] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

[0046] Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

[0047] Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

[0048] Additionally, while terms such as "first," "second," etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0049] Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

[0050] Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

[0051] In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0052] In this specification, a block can be variously expressed as a unit, an area, a unit, a partition, etc., and a sample can be variously expressed as a pixel, a pel, a pixel, etc.

[0053] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

[0054] FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

[0055] Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

[0056] The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Hereinafter, in the embodiments of the present disclosure, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

[0057] A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and / or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

[0058] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter prediction, and an intra prediction unit (125) that performs intra prediction or intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by the entropy encoding unit (165) and transmitted to the decoder.

[0059] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0060] The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1 / 8 pixels.

[0061] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode can be used as motion prediction methods.

[0062] The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

[0063] Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

[0064] In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

[0065] The quantization unit (135) can quantize values ​​converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

[0066] The rearrangement unit (160) can perform rearrangement of coefficient values ​​for quantized residual values.

[0067] The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

[0068] The entropy encoding unit (165) can perform entropy encoding based on the values ​​produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present disclosure, it is possible to signal and transmit information indicating that motion information is derived and used on the decoder side and information on a technique used to derive motion information.

[0069] The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values ​​quantized in the quantization unit (135) and inversely transform the values ​​transformed in the transformation unit (130). The residual values ​​generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0070] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset from the original image on a pixel-by-pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

[0071] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

[0072] FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

[0073] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0074] When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

[0075] The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

[0076] The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

[0077] The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

[0078] The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0079] The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

[0080] The prediction unit (230, 235) can generate a prediction block based on prediction block generation related information provided from the entropy decoding unit (210) and previously decoded block or picture information provided from the memory (245).

[0081] As described above, when performing intra prediction or intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

[0082] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the encoder (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (230) performs prediction based on the information transmitted from the encoder (100).

[0083] The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the subsequent picture of the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided by the image encoder. In order to perform inter prediction, it can be determined based on the encoding unit whether the motion prediction method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode.

[0084] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder.

[0085] The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a unit that performs filtering on the reference pixels of the current block and can determine whether to apply the filter based on the prediction mode of the current prediction unit and apply it. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0086] The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

[0087] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0088] Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from a video encoding device, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0089] The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and information on the offset value. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

[0090] The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

[0091] Figure 3 illustrates a process for generating a prediction block of a current block according to the present disclosure.

[0092] Referring to Figure 3, an appearance histogram of the current block can be constructed (S300).

[0093] Hereinafter, the method of constructing the appearance histogram of the current block will be examined with reference to FIGS. 4 to 6.

[0094] FIG. 4 is a diagram showing an example of a method for constructing an appearance histogram of a current block according to the present disclosure.

[0095] The histogram of occurrence of the current block can be constructed based on the intra prediction mode of the reference block. In this case, the reference block may be a pre-reconstructed block. In the occurrence histogram of the present invention, the x-axis may represent an intra prediction mode (IPM), and the y-axis may represent an amplitude value corresponding to each IPM. The intra prediction mode (IPM) of the x-axis may be constructed based on the intra prediction mode of the pre-reconstructed reference block. The amplitude value of the y-axis is a value that numerically expresses the degree to which each IPM contributes to the prediction of the current block, and a high amplitude value may indicate that the relative importance of the corresponding IPM is high.

[0096] For example, when the appearance histogram is configured based on the intra prediction mode or size of the reference block, the position on the x-axis of the histogram may represent the intra prediction mode index of the reference block, and the y-axis may be an amplitude value calculated based on the size of the reference block. Here, the amplitude value may be calculated as the product of the width and the height of the reference block. For example, when the intra prediction mode index of the reference block is 20, and the width and the height are each 8, a graphic element having an amplitude value of 64 (width 8 x height 8) may be added to the position x = 20 of the appearance histogram.

[0097] FIG. 5 is a diagram showing another example of a method for constructing an appearance histogram of a current block according to the present disclosure.

[0098] Referring to FIG. 5, if a reference block has multiple intra prediction modes, the appearance histogram of the current block may include amplitude values ​​corresponding to each intra prediction mode. The amplitude values ​​may be calculated based on the size of the reference block, and specifically, may be calculated as the product of the width and height of the reference block.

[0099] For example, if a reference block is in any one of decoder-side intra mode derivation (DIMD) mode, template-based intra mode derivation (TIMD) mode, or spatial geometric partitioning mode (SGPM), a plurality of graphic elements may be added to an appearance histogram based on amplitude values ​​corresponding to each of a plurality of intra prediction modes included in the mode. In this case, the number of the plurality of intra prediction modes may be 5 if the reference block is in DIMD mode, 2 if the reference block is in TIMD mode, and 2 if the reference block is in SGPM.

[0100] Meanwhile, even if the reference block has multiple intra prediction modes, only some of the multiple intra prediction modes may have amplitude values ​​corresponding to them and be included in the appearance histogram. For example, if the reference block is in DIMD mode, only graphic elements based on amplitude values ​​corresponding to some (e.g., 1) of the multiple intra prediction modes (e.g., 5) may be added to the appearance histogram.

[0101] FIG. 6 is a diagram showing examples of reference blocks of a current block for configuring an appearance histogram of the current block according to the present disclosure.

[0102] Referring to FIG. 6, not only reference blocks spatially adjacent to the current block located at the center, but also non-adjacent reference blocks located a certain distance away can be utilized to construct a histogram of the current block. The positions and numbers of reference blocks illustrated in FIG. 6 are merely exemplary, and the arrangement of the reference blocks may vary depending on various conditions, such as the position and size of the current block, and whether surrounding reference blocks are restored. Therefore, the present invention is not limited to the specific arrangement of the reference blocks illustrated in FIG. 6, and various arrangements may be included in the technical spirit of the present disclosure.

[0103] Additionally, if the reference block of the current block is reconstructed based on at least one of the matrix-based intra prediction (MIP) mode, the intra template matching prediction (intra TMP) mode, or the intra block copy (IBC) mode, the reference block may not be considered in constructing the appearance histogram of the current block.

[0104] Next, according to the present invention, the accuracy of intra prediction can be improved by correcting the amplitude value of the appearance histogram. The details of this are as follows.

[0105] The amplitude value of the y-axis corresponding to the IPM corresponding to the x-axis of the appearance histogram of the current block can be calculated based on the size of the reference block, and the calculated amplitude value can be corrected based on the distance between the current block and the reference block. Hereinafter, the 'correction distance' is the distance between the current block and the reference block, and may mean distance information used for correcting the amplitude value of the appearance histogram. Here, the correction distance can be calculated based on a reference point within each block. The reference point may include at least one of a center sample, an upper left sample, an upper right sample, a lower left sample, or a lower right sample located within each block. In addition, the current block and the reference block may have reference points at different positions depending on the position of each block or the relative positional relationship between each block.

[0106] For example, the correction distance may be calculated using at least one of the L1 distance (Manhattan distance) or the L2 distance (Euclidean distance) between the reference point of the current block and the reference point of the reference block. The L1 distance may be calculated by calculating the absolute values ​​of the horizontal distance difference and the vertical distance difference between the reference point of the current block and the reference point of the reference block, respectively, and then taking the sum of the absolute values. On the other hand, the L2 distance may be calculated by squaring the horizontal distance difference and the vertical distance difference between the reference point of the current block and the reference block, respectively, and then taking the square root of the sum thereof. That is, the L1 distance may be defined as the accumulated value of the axial distance between reference points, and the L2 distance may be defined as the straight-line distance between reference points.

[0107] Alternatively, the correction distance may be calculated as an average value of the L1 distance between a plurality of reference points existing at the same location within the current block and the reference block. For example, the average value of at least two of the L1 distance between the center sample of each of the current block and the reference block, the L1 distance between the upper right samples, the L1 distance between the lower right samples, the L1 distance between the upper left samples, or the L1 distance between the lower left samples may be utilized as the correction distance. Similarly, the correction distance may be calculated based on the L2 distance, or may be calculated through a combination of the L1 distance and the L2 distance.

[0108] As another example, the correction distance can be derived through the cosine similarity between a vector derived based on a reference point of the current block and a vector derived based on a reference point of the reference block. Here, the cosine similarity is a value that numerically expresses the directional similarity of two vectors, and can be calculated as a value dividing the inner product of the two vectors by the product of the magnitudes of each vector. In general, the closer the cosine similarity value is to 1, the more similar the directions of the two vectors are, and the closer it is to 0 or a lower value, the more different the directions may be. As a specific example, the vector of the current block can be defined based on the difference between a reference point within the current block and a predetermined reference point other than the reference point, and the vector of the reference block can also be defined in a similar manner. The correction distance can be derived based on the cosine similarity between the vector of the current block and the vector of the reference block.

[0109] As another example, a predetermined pair of reference points may be set among a plurality of reference points existing in each of the current block and the reference block, and the correction distance may be calculated through a weighted sum based on the distance between the set plurality of reference point pairs. Here, the predetermined pair of reference points may be a pair between reference points existing at the same location within each block. In addition, the weights used in the weighted sum may be determined based on the distance between each pair of reference points.

[0110] The above correction distance may not be calculated depending on certain conditions. For example, if at least one block among the current block or the reference block is smaller than a certain size, the calculation of the correction distance may be omitted. The certain size may be set as NxM, where N and M may each be a natural number greater than or equal to 1, and may have the same value.

[0111] Alternatively, whether to perform the correction distance calculation process may be determined based on the prediction method used to restore the reference block. For example, if the reference block is not restored based on intra prediction or if it is restored using a predetermined intra prediction method, the correction distance calculation process may be omitted. The predetermined intra prediction method may include at least one of a block vector-based intra prediction method, an intra prediction method based on a single intra prediction mode, or an intra prediction method based on multiple intra prediction modes (e.g., a DIMD mode, a TIMD mode, etc.).

[0112] As another example, if the correction distance for a preceding reference block is greater than or equal to a predetermined reference value N, the correction distance calculation process for subsequent reference blocks may be omitted. Here, N may be a natural number greater than or equal to 1. In addition, the correction distance may be derived based on the above-described L1 distance, L2 distance, or cosine similarity.

[0113] As another example, whether or not to perform the correction distance calculation process may be determined based on the positional relationship between the current block and the reference block. As a specific example, if the reference block corresponds to at least one of a spatially adjacent block, a spatially non-adjacent block, a temporally adjacent block, a temporally non-adjacent block, or a history-based block of the current block, the correction distance calculation process may be omitted.

[0114] Below, a specific method for correcting the amplitude value using the above correction distance is described.

[0115] The amplitude value of the appearance histogram of the current block can be corrected based on the distance (correction distance) between the current block and the reference block. For example, the amplitude value can be corrected by multiplying the amplitude value calculated based on the size of the reference block by a correction coefficient calculated based on the correction distance. The value of the correction coefficient (Scale) can be in the real number range from 0 to 1, and the value range can be restricted or expanded according to certain conditions. As a specific example, when the range of the correction coefficient value is restricted to the real number range from 0.5 to 1, the correction coefficient can be calculated according to the following mathematical expression 1.

[0116]

[0117] In mathematical expression 1, Scale can represent a correction coefficient, CurrDist can represent a distance (correction distance) between the current block and the reference block, and MaxDist can represent a distance between the current block and the reference block at the maximum distance that can be referenced.

[0118] When the above correction coefficient is calculated, the amplitude value can be corrected by multiplying the correction coefficient by the amplitude value calculated based on the size of the reference block according to the following mathematical expression 2.

[0119]

[0120] In mathematical expression 2, Ampl modified can mean the corrected amplitude value, and Ampl initial may refer to an initial amplitude value calculated before correction, i.e. based on the size of the reference block. The corrected amplitude value may be added to the appearance histogram of the current block instead of the amplitude value before correction.

[0121] The above process of correcting the initial amplitude value based on the correction coefficient may be determined whether or not to be performed according to certain conditions. For example, if the distance (correction distance) between the current block and the reference block has not been calculated, the performance of the correction process may be omitted. As another example, whether or not to be performed may be determined based on the positional relationship between the current block and the reference block. As a specific example, if the reference block corresponds to at least one of a spatially adjacent block, a spatially non-adjacent block, a temporally adjacent block, a temporally non-adjacent block, or a history-based block of the current block, the performance of the correction process may be omitted.

[0122] As another example, whether or not to perform the correction process may be determined based on the prediction method used to restore the reference block. For example, if the reference block is restored based on a block vector or using a prediction method based on G or more (or fewer) intra prediction modes, the correction process may be omitted. Here, G may be a natural number greater than or equal to 1.

[0123] As another example, if the correction distance for the preceding reference block is greater than or equal to a predetermined reference value P, the process of correcting the amplitude values ​​of subsequent reference blocks may be omitted. Here, P may be a natural number greater than or equal to 1. In addition, the correction distance may be derived based on the above-described L1 distance, L2 distance, or cosine similarity.

[0124] Referring to FIG. 3, the intra prediction mode of the current block can be derived based on the appearance histogram of the current block (S310).

[0125] For example, among the intra prediction modes included in the appearance histogram of the current block, up to N intra prediction modes can be derived by sorting them in ascending order of amplitude value. Here, the amplitude value may mean a value calculated based on the size of the reference block, or may mean an amplitude value corrected based on a correction distance. In addition, N may be a natural number greater than or equal to 1.

[0126] Referring to FIG. 3, a prediction block of the current block can be generated based on the derived intra prediction mode (S320).

[0127] Below, the process of generating a prediction block of the current block will be examined in detail with reference to Fig. 7.

[0128] FIG. 7 is a diagram illustrating an example of a method for generating a prediction block of a current block according to the present disclosure.

[0129] Referring to FIG. 7, at most S intra prediction modes can be derived from an occurrence histogram of a current block. Here, S may be a natural number greater than or equal to 1. A prediction block of the current block can be generated through a weighted sum between prediction blocks generated based on the derived intra prediction modes and a planar mode, respectively. At this time, a prediction block generated based on a block vector (BV)-based mode instead of the planar mode can be used for the weighted sum. The block vector-based mode may include an intra template matching prediction (intra TMP) mode or an intra block copy (IBC) mode. Additionally, the block vector-based mode may be derived from a pre-restored reference block located around the current block. The pre-restored reference block may include at least one of a reference block adjacent to the current block that has been sub / decoded or a non-adjacent reference block that has been sub / decoded.

[0130] For example, the weight of a prediction block corresponding to an induced intra prediction mode can be calculated based on the amplitude value of the occurrence histogram corresponding to each intra prediction mode index, and the weight of a prediction block corresponding to a planar mode or a block vector-based mode can have a fixed value. For example, when the number of induced intra prediction modes is 5, the weight can be defined according to the following mathematical expressions 3 and 4.

[0131]

[0132]

[0133] In mathematical expression 3, weight(M i ) may mean a weight applied to a prediction block generated by any one of the above-described intra prediction modes, and ampl(M i) may mean the amplitude value on the appearance histogram of the corresponding mode. Here, i may be any one of 1, 2, 3, 4, or 5.

[0134] Also, in Equation 4, weight(M planar / BV ) may mean a fixed weight of a prediction block corresponding to a planar mode or a block vector (BV) based mode applied to the above weighted sum.

[0135] As another example, if there is an amplitude value of a mode corresponding to an intra prediction mode index 0 (x-axis = 0) in the appearance histogram of the current block, a weight of a prediction block corresponding to the planar mode or block vector-based mode can be calculated based on the amplitude value. Here, the intra prediction mode index 0 can be, for example, a non-directional intra prediction mode such as a planar mode or a block vector-based mode. As a more specific example, if the number of derived intra prediction modes is 5, the weight can be calculated as in the following mathematical expressions 5 and 6.

[0136]

[0137]

[0138] In mathematical expression 5, weight(M j ) may mean a weight applied to a prediction block generated by one of the above-described intra prediction modes, and in Equations 5 and 6, ampl(M j ) may mean an amplitude value on the appearance histogram of the corresponding mode. Here, j may be any one of 1, 2, 3, 4, or 5. In addition, ampl(M0) may mean an amplitude value on the appearance histogram of intra prediction mode index 0.

[0139] Additionally, in Equation 6, weight(M planar / BV) may refer to the weight of the prediction block corresponding to the planar mode or block vector (BV)-based mode applied to the above weighted sum. That is, the weight of the prediction block corresponding to the planar mode or block vector-based mode may be calculated based on the amplitude value corresponding to the intra prediction mode 0 of the appearance histogram, rather than a fixed value.

[0140] On the other hand, if the amplitude value of the mode corresponding to the intra prediction mode 0 does not exist in the appearance histogram of the current block, the weight of the prediction block corresponding to the planar mode or block vector-based mode may have a fixed value H. Here, H may be a real value greater than or equal to 0 and less than or equal to 1. Alternatively, the weight may be , and the above R may be a natural number greater than or equal to 1 and less than or equal to 64.

[0141] As another example, if a mode corresponding to an intra prediction mode index 0 is included in the maximum T intra prediction modes derived by sorting the intra prediction modes included in the appearance histogram of the current block in ascending order of amplitude value, the prediction block of the current block can be generated through a weighted sum between prediction blocks generated respectively based on the derived intra prediction modes. Here, the intra prediction mode index 0 may be, for example, a non-directional intra prediction mode such as a planar mode or a block vector-based mode. In addition, T may be a natural number greater than or equal to 1.

[0142] The weights of the prediction blocks corresponding to each derived intra prediction mode applied to the above weighted sum can be calculated based on the amplitude values ​​in the appearance histogram of each mode. For example, if the number of derived intra prediction modes is six, the weights can be calculated as shown in the following mathematical expression (7).

[0143]

[0144] In mathematical expression 7, weight(Mk ) may mean a weight applied to a prediction block generated by any one of the above-described intra prediction modes, and ampl(M k ) may mean an amplitude value on the appearance histogram of the corresponding mode. In particular, ampl(M0) may mean an amplitude value on the appearance histogram of intra prediction mode index 0. Here, k may be any one of 0, 1, 2, 3, 4, or 5.

[0145] Alternatively, when the number of derived intra prediction modes is L and the amplitude value of the mode corresponding to the intra prediction mode index 0 exists in the appearance histogram of the current block, the weight of the prediction block corresponding to the planar mode or block vector-based mode and the weight of the derived intra prediction mode may be calculated as in the following mathematical expressions 8 and 9.

[0146]

[0147]

[0148] In mathematical expression 8, weight(M planar / BV ) may mean the weight of the prediction block corresponding to the planar mode or block vector (BV) based mode applied to the above weighted sum. In addition, ampl(M0) may mean the amplitude value on the appearance histogram of the intra prediction mode index 0. The Clip3 function may be a function that determines an output value based on three input arguments. Specifically, the Clip3 function may be a function that determines an output value based on the first input argument, the minimum value (weight min ), the second input argument is the maximum value (weight max), and the third input value is received as input, and if the input value is less than the minimum value, the minimum value is output, if the input value is greater than the maximum value, the maximum value is output, and if the input value is located between the minimum value and the maximum value, the input value can be output as is. Here, L may mean the number of intra prediction modes derived from the appearance histogram, and may be a natural number greater than or equal to 1. In addition, weight min and weight mzx can be a real number greater than or equal to 0 and less than or equal to 1, and weight min Silver weight mzx may be less than or equal to the weight. For example, the above weight min can be 0, and weight mzx can be 1.

[0149] In mathematical expression 9, weight(M x ) is one of the derived intra prediction modes (M x ) may mean a weight applied to a prediction block generated by x. Here, x may be a natural number greater than or equal to 1 and less than or equal to L.

[0150] When performing a weighted sum to generate a prediction block of the current block, the weights of the prediction block corresponding to each derived intra prediction mode and the prediction block corresponding to the planar mode or block vector mode can be derived based on the appearance histogram of the current block. At this time, if an amplitude value for intra prediction mode 0 exists in the appearance histogram, the weights may also be derived based on the amplitude value.

[0151] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0152] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general processors, controllers, microcontrollers, microprocessors, etc.

[0153] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to various embodiments of the present disclosure to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0154] The present disclosure can be industrially utilized in the fields of methods, devices, and recording media for video encoding / decoding.

Claims

1. A step of generating an appearance histogram of the current block based on a reference block of the current block; A step of deriving one or more intra prediction modes for the current block based on the appearance histogram; and A step of generating a prediction block of the current block based on the derived intra prediction mode, A method for decoding an image, wherein the above appearance histogram is composed of an intra prediction mode and an amplitude value of the reference block.

2. In paragraph 1, An image decoding method, wherein, when the reference block has a plurality of intra prediction modes, the appearance histogram includes an amplitude value corresponding to at least one of the plurality of intra prediction modes.

3. In paragraph 1, A method for decoding an image, wherein the amplitude value is determined based on the size of the reference block.

4. In paragraph 3, The above determined amplitude value is corrected based on the distance between the current block and the reference block, The above distance is calculated based on the reference points of the current block and the reference block, A method for decoding an image, wherein the reference point includes at least one of samples located at the center, upper left, upper right, lower left, or lower right within the current block and the reference block.

5. In paragraph 4, A method for decoding an image, wherein the above distance is calculated by weighting distances between a plurality of reference point pairs set based on a plurality of reference points of the current block and a plurality of reference points of the reference block.

6. In paragraph 1, First and second prediction blocks are generated based on the above-described intra prediction mode and the predetermined mode, respectively. A method for decoding an image, wherein the prediction block of the current block is generated through a weighted sum between the first prediction block and the second prediction block.

7. In paragraph 6, A method for decoding an image, wherein the above-described mode includes at least one of a planar mode or a block vector-based mode.

8. In paragraph 6, The weights for the weighted sum include at least one of a first weight for the first prediction block or a second weight for the second prediction block, The above first weight is derived based on the amplitude value, A method for decoding an image, wherein the second weight is a fixed value.

9. In paragraph 6, An image decoding method, wherein, when an amplitude value of a mode corresponding to an intra prediction mode index 0 exists in the above appearance histogram, a weight for the weighted sum is calculated based on the amplitude value of the derived intra prediction mode and the amplitude value of the mode corresponding to the intra prediction mode index 0.

10. In paragraph 1, A method for decoding an image, wherein, if the derived intra prediction mode includes a mode corresponding to an intra prediction mode index of 0, the prediction block of the current block is generated through a weighted sum between prediction blocks generated based on each of the derived intra prediction modes.

11. A step of generating an appearance histogram of the current block based on the reference block of the current block; A step of deriving one or more intra prediction modes for the current block based on the appearance histogram; and A step of generating a prediction block of the current block based on the derived intra prediction mode, A method for encoding an image, wherein the above appearance histogram is composed of an intra prediction mode and an amplitude value of the reference block.

12. A computer-readable storage medium for storing a bitstream generated by the image encoding method according to Article 11.

13. A method for transmitting a bitstream generated by a video encoding method according to Article 11.

Images