Method and device for intra directional prediction through adaptive filtering

Abstract

An image encoding method and an image decoding method, according to the present disclosure, may comprise: determining an intra prediction mode for the current block; determining a reference sample to be referred to during intra prediction of the current block; and performing intra prediction on the current block on the basis of the determined intra prediction mode and the reference sample.

Description

Method and device for predicting in-frame orientation through adaptive filtering

[0001] The present invention relates to an image encoding / decoding method and apparatus based on adaptive filtering and intra-frame directionality prediction.

[0002] Market demand for high-resolution video is increasing, and consequently, there is a need for technology that can efficiently compress high-resolution video. In response to these market demands, the Moving Picture Expert Group (MPEG) of ISO / IEC and the Video Coding Expert Group (VCEG) of ITU-T jointly formed the Joint Collaborative Team on Video Coding (JCT-VC), completed the development of the High Efficiency Video Coding (HEVC) video compression standard in January 2013, and have been actively conducting research and development on next-generation compression standards.

[0003] Video compression is broadly composed of intra-frame prediction (or intra prediction), inter-frame prediction (or inter prediction), transformation, quantization, entropy coding, and in-loop filters. Meanwhile, along with the increasing demand for high-resolution video, the demand for stereoscopic video content as a new video service is also increasing. Discussions are underway regarding video compression technologies to effectively provide high-resolution and ultra-high-resolution stereoscopic video content.

[0004] The present disclosure aims to provide an image decoding method and apparatus based on intra-prediction.

[0005] The present disclosure aims to provide a method and apparatus for determining an intra-predicted mode based on a mode index, a base angle, and a delta angle.

[0006] The present disclosure aims to provide a method and apparatus for determining a reference sample based on an intra-prediction mode.

[0007] The present disclosure aims to provide a method and apparatus for selecting an interpolation filter based on the frequency components of a reference sample.

[0008] The present disclosure aims to provide a method and apparatus for interpolating filtering of reference samples based on a plurality of tap filters and integer position pixels within a reference line.

[0009] The present disclosure aims to provide an intra prediction method and apparatus based on a determined intra prediction mode and a determined reference sample.

[0010] The present disclosure aims to provide an intra-bidirectional prediction method and apparatus based on a current block partitioning method and a weighting method for each partitioned area.

[0011] The image decoding method and apparatus according to the present disclosure can determine an intra prediction mode for a current block, determine a reference sample of the current block, and perform intra prediction for the current block.

[0012] In the image decoding method and apparatus according to the present disclosure, the step of determining the reference sample may include the step of determining a reference line and filtering the reference line.

[0013] In the image decoding method and apparatus according to the present disclosure, the reference line is determined based on at least one of an available reference line or a multi-line index, and the reference line filtering may include at least one of corner filtering or smoothing filtering.

[0014] In the image decoding method and apparatus according to the present disclosure, the corner filtering may be performed based on at least one of the left reference sample or the top reference sample.

[0015] In the image decoding method and apparatus according to the present disclosure, the intra prediction for the current block includes intra directional interpolation filtering, and the intra directional interpolation filtering may be performed based on at least one of an intra prediction mode or frequency information of the reference sample.

[0016] In the image decoding method and apparatus according to the present disclosure, the frequency information of the reference sample may include at least one of the frequency component of the reference sample or the correlation of frequency components between the reference samples.

[0017] In the image decoding method and apparatus according to the present disclosure, the intra prediction includes intra bidirectional prediction, and the intra bidirectional prediction may be performed based on at least one of the reference sample or weight.

[0018] In the image decoding method and apparatus according to the present disclosure, the reference sample includes a reference sample of the left reference line of the current block or a reference sample of the top reference line of the current block, and the weight may be derived based on the distance between the predicted sample and the reference sample.

[0019] In the image decoding method and apparatus according to the present disclosure, the weight can be corrected for each section in which the current block is divided into a plurality of sections.

[0020] In the image decoding method and apparatus according to the present disclosure, the method for dividing the current block can be determined based on the directionality of the intra prediction mode.

[0021] The image encoding method and apparatus according to the present disclosure can determine an intra prediction mode for a current block, determine a reference sample of the current block, and perform intra prediction for the current block.

[0022] A non-transient computer-readable digital storage medium is provided, which stores video / image information generated according to the image encoding method according to the present disclosure.

[0023] A method and apparatus for transmitting video / image information generated according to the image encoding method according to the present disclosure are provided.

[0024] According to the present disclosure, an intra-prediction mode and a reference sample of the current block can be determined to perform a prediction for the current block.

[0025] According to the present disclosure, the intra prediction mode of the current block can be determined by signaling mode index and angle information from a bitstream.

[0026] According to the present disclosure, a reference line and a reference sample can be determined based on intra-prediction mode information.

[0027] According to the present disclosure, interpolation filtering can be performed on a reference line.

[0028] According to the present disclosure, the interpolation filter used for the interpolation filtering may be derived based on reference line information or pixels at integer positions within the reference line.

[0029] According to the present disclosure, whether to perform an intra-bidirectional prediction mode can be determined based on the intra-prediction mode.

[0030] According to the present disclosure, the current block can be divided into a plurality of zones, and intra-bidirectional prediction can be performed for each prediction sample in a specific zone.

[0031] FIG. 1 is a block diagram showing an image encoding device according to the present invention.

[0032] FIG. 2 is a block diagram showing an image decoding device according to the present invention.

[0033] Figure 3 illustrates an intra prediction method performed in the image decoder of the present invention.

[0034] FIG. 4 illustrates all directions of the directional mode used in the decoding process according to the present invention.

[0035] Figure 5 illustrates the samples that serve as reference targets for filtering when performing corner filtering.

[0036] Figure 6 illustrates the directionality of the intra prediction mode.

[0037] Figure 7 shows that the decoder performs intra-bidirectional prediction on the current block.

[0038] Figure 8 illustrates an example of dividing the current block into three zones.

[0039] FIG. 9 is a block diagram illustrating an image encoding method according to the present disclosure.

[0040] Embodiments of the present invention are described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0041] Throughout this specification, when a part is described as being 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with other elements in between.

[0042] Furthermore, throughout this specification, when a part is described as 'comprising' a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0043] Additionally, terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0044] Additionally, in the embodiments relating to the device and method described herein, some components of the device or some steps of the method may be omitted. Also, the order of some components of the device or some steps of the method may be changed. Additionally, other components or other steps may be inserted into some components of the device or some steps of the method.

[0045] In addition, some components or steps of each embodiment of the present invention may be added to or replace some components or steps of other embodiments of the present invention.

[0046] Furthermore, the components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, for convenience of explanation, each component is described by listing it as a separate component, and at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separated embodiments of each component are also included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0047] First, the terms used in this application are briefly explained as follows.

[0048] The video decoding apparatus described below may be a device included in a civilian security camera, civilian security system, military security camera, military security system, personal computer (PC), notebook computer, portable multimedia player (PMP), wireless communication terminal, smartphone, and server terminal such as a TV application server and service server; it may also refer to various devices equipped with a user terminal such as various devices, a communication device such as a communication modem for performing communication with a wired or wireless communication network, a memory for storing various programs and data for decoding video or predicting between or within a screen for decoding, and a microprocessor for executing programs to perform calculations and control.

[0049] In addition, the video encoded into a bitstream by the encoder can be transmitted to a video decoding device in real-time or non-real-time via wired or wireless communication networks such as the Internet, local area wireless networks, wireless LAN networks, WiBro networks, and mobile communication networks, or via various communication interfaces such as cables and Universal Serial Bus (USB), to be decoded, restored as a video, and played back. Alternatively, the bitstream generated by the encoder may be stored in memory. The memory may include both volatile and non-volatile memory. In this specification, memory may be described as a recording medium that stores the bitstream.

[0050] Typically, a video can be composed of a series of pictures, and each picture can be divided into coding units such as blocks. Furthermore, those skilled in the art to which this embodiment belongs will understand that the term "picture" described below may be replaced with other terms having equivalent meanings, such as "image" or "frame."

[0051] As used in this specification, the term "current block" refers to a spatial unit that constitutes all or part of the reconstructed samples generated during the image restoration process, and signifies a basic unit where in-frame directionality prediction and adaptive filtering are performed. Furthermore, those skilled in the art to which this embodiment belongs will understand that the term "coding unit" may be replaced with other terms having equivalent meanings, such as unit block, block, etc.

[0052] Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, redundant descriptions of identical components are omitted.

[0053] FIG. 1 is a block diagram showing an image encoding device according to the present invention.

[0054] Referring to FIG. 1, the image encoding device (100) may include a picture splitting unit (110), a prediction unit (120, 125), a conversion unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse conversion unit (145), a filter unit (150), and a memory (155).

[0055] The picture splitting unit (110) can split the input picture into at least one processing unit. At this time, the processing unit may be a Prediction Unit (PU), a Transform Unit (TU), or a Coding Unit (CU). In the following embodiments of the present invention, the term "coding unit" may be used to mean a unit that performs coding or a unit that performs decoding.

[0056] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter-frame prediction, and an intra prediction unit (125) that performs intra prediction or intra-frame prediction. It may determine whether to use inter prediction or perform intra prediction for a prediction unit, and determine specific information according to each prediction method (e.g., intra prediction mode, motion vector, reference picture, etc.). The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder.

[0057] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a partially encoded area within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0058] In the reference picture interpolation unit, reference picture information is received from memory (155), and pixel information of integer pixels or less can be generated from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 4 pixel units. In the case of chrominance signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 8 pixel units.

[0059] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to calculate motion vectors. Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various motion prediction methods such as the Skip method, Merge method, TMVP (Temporal Motion Vector Prediction) method, SMVP (Spatial Motion Vector Prediction) method, and Intra Block Copy method can be used.

[0060] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block that has performed inter prediction, and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction can be replaced with the reference pixel information of the surrounding intra prediction block. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced with at least one of the available reference pixels.

[0061] Additionally, a residual block can be generated that includes residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit. The generated residual block can be input to the conversion unit (130).

[0062] In the transformation unit (130), the residual block containing residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra-prediction mode information of the prediction unit used to generate the residual block.

[0063] The quantization unit (135) can quantize the values ​​converted into the frequency domain in the conversion unit (130). The quantization coefficient may vary depending on the block or the importance of the image. The values ​​produced by the quantization unit (135) may be provided to the inverse quantization unit (140) and the reordering unit (160).

[0064] The reordering unit (160) can perform reordering of coefficient values ​​for quantized residual values.

[0065] The reordering unit (160) can convert two-dimensional block-shaped coefficients into one-dimensional vector forms through a coefficient scanning method. For example, the reordering unit (160) can convert the coefficients from DC to high-frequency range coefficients into one-dimensional vector forms by scanning using a Zig-Zag Scan method. Depending on the size of the conversion unit and the intra-prediction mode, a vertical scan that scans two-dimensional block-shaped coefficients in the column direction or a horizontal scan that scans two-dimensional block-shaped coefficients in the row direction may be used instead of the Zig-Zag Scan. That is, depending on the size of the conversion unit and the intra-prediction mode, it can be determined whether to use a Zig-Zag Scan, a vertical scan, or a horizontal scan.

[0066] The entropy encoding unit (165) can perform entropy encoding based on the values ​​calculated by the reordering unit (160). Entropy encoding can use various encoding methods, such as, for example, Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the reordering unit (160) and the prediction unit (120, 125). Additionally, according to the present invention, it is possible to signal and transmit information indicating that motion information is derived and used in the decoder side, and information regarding the technique used for deriving motion information.

[0067] In the inverse quantization unit (140) and inverse transformation unit (145), the values ​​quantized in the quantization unit (135) are inverse quantized, and the values ​​transformed in the transformation unit (130) are inverse transformed. The residual value generated in the inverse quantization unit (140) and inverse transformation unit (145) can be combined with the prediction unit predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0068] The filter unit (150) may include at least one of a loop filter, a constrained directional enhancement filter, a cross-component sample offset correction unit, and a loop restoration unit. The loop filter may receive a signal predicted by the prediction unit (120, 125) and a residual signal from the inverse transform unit (145) as inputs, and output a restored signal after the loop filter has been performed. The constrained directional enhancement filter may remove ringing phenomena while maintaining image clarity by considering edge information of the restored signal. The constrained directional enhancement filter may calculate the direction of the sample to find the direction of the edge and selectively remove noise according to the edge direction. The cross-component sample offset correction unit may restore color difference based on high-resolution information of the provided luminance sample. The loop restoration unit may be performed through a super-resolution and loop restoration process. Upscaling of the sample is performed through the super-resolution process, and distortion and texture loss spread throughout the sample can be corrected through the loop restoration process.

[0069] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter-prediction.

[0070] FIG. 2 is a block diagram showing an image decoding device according to the present invention.

[0071] Referring to FIG. 2, the image decoder (200) may include an entropy decoder (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0072] When a video bitstream is input to a video encoder, the input bitstream can be decoded using the reverse procedure of the video encoder.

[0073] The entropy decoding unit (210) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit of the video encoder performed. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the video encoder.

[0074] The entropy decoding unit (210) can decode information related to intra-prediction and inter-prediction performed in the encoder.

[0075] The reordering unit (215) can perform reordering based on the method of reordering the entropy-decoded bitstream in the encoding unit in the entropy decoding unit (210). The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them to the form of two-dimensional block coefficients.

[0076] The inverse quantization unit (220) can perform inverse quantization based on the coefficient values ​​of the rearranged block and the quantization parameters provided by the encoder.

[0077] The inverse transform unit (225) can perform inverse transforms, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms, i.e., DCT, DST, and KLT, performed by the transform unit on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transformation technique (e.g., DCT, DST, KLT) can be selectively performed according to multiple pieces of information such as a prediction method, the size of the current block, and the prediction direction.

[0078] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0079] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit (210), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit in the current encoding unit, and determines whether the prediction unit performs inter prediction or intra prediction.

[0080] The inter prediction unit (230) can perform inter prediction for the current prediction unit based on information included in at least one picture, either a previous picture or a subsequent picture, of the current picture containing the current prediction unit, using information required for inter prediction of the current prediction unit provided by the video encoder. To perform inter prediction, based on the encoding unit, it can determine whether the motion prediction method of the prediction unit included in the corresponding encoding unit is a Skip Mode, Merge Mode, SMVP / TMVP Mode, or Intra Block Copy Mode.

[0081] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, it can perform intra prediction based on the intra prediction mode information of the prediction unit provided by the image encoder. The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixel of the current block using the prediction mode of the prediction unit and the AIS filter information provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0082] The reference pixel interpolation unit can generate a reference pixel of an integer value or less by interpolating the reference pixel when the prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on the pixel value interpolated from the reference pixel. If the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is DC mode.

[0083] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include at least one of a loop filter, a Constrained Directional Enhancement Filter (CDEF), a Cross-Component Sample Offset Correction Unit (CCSO), and a loop restoration unit.

[0084] The decoder can receive information from the encoder regarding whether a loop filter has been applied to the corresponding block or picture, and if so, whether a strong or weak filter has been applied. The encoder's loop filter receives loop filter-related information provided by the video encoder, and the video decoder can perform loop filtering on the corresponding block.

[0085] The offset correction unit can perform offset correction on the restored image based on the type of offset correction and offset value information applied to the image during encoding. CDEF and CCSO can be applied to the decoding unit based on information regarding whether CDEF was applied, CDEF coefficient information, and CCSO information provided by the encoder.

[0086] The memory (245) can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to the output unit.

[0087] Figure 3 illustrates an intra prediction method performed in the image decoder of the present invention.

[0088] Referring to FIG. 3, the process of performing intra prediction for the current block may include the step of determining the intra prediction mode of the current block (S300), the step of determining a reference sample for the intra prediction of the current block (S310), and the step of performing the intra prediction of the current block (S320).

[0089] The current prediction block (current block) and the restoration block adjacent to the current block can be defined by various partitioning methods. These various partitioning methods may include binary partitioning, binary partitioning, quad partitioning, etc. As an example of quad partitioning, various partitioning ratios such as 1:2:1 or 1:4:2:1 may be used. The partitioning of the blocks may be performed recursively, and the current block may be partitioned repeatedly until the quad tree shape reaches a constrained shape. The minimum unit pixel number of the block may vary depending on the color component of the current block. For example, the minimum unit pixel number of the luminance block and the chrominance block may be 4 pixels and 2 pixels, respectively. The superblock may be defined as 128*128, but is not limited thereto and may be 256*256 or larger. The aspect ratio of the superblock may be 8:1 or larger, or 1:8 or larger.

[0090] Intra-prediction mode determination step

[0091] Based on the information (directional mode) signaled through the bitstream, it can be determined whether the intra prediction mode of the current block is a directional mode. The directional mode can indicate whether the intra prediction mode of the current block is a directional mode. For example, if the value of the directional mode is 1, it may indicate that the intra prediction mode of the current block is a directional mode, and if not, it may indicate that the intra prediction mode of the current block is a non-directional mode.

[0092] If the intra prediction mode of the current block is a directional mode, the direction of the directional mode can be determined based on angle information signaled from the bitstream. The angle information may include at least one of information regarding a nominal angle or information regarding a delta angle.

[0093] FIG. 4 illustrates the directionality of a directionality mode according to the present disclosure. In FIG. 4, the directionality of D45_PRED, D67_PRED, V_PRED, D113_PRED, D135_PRED, D157_PRED, H_PRED, and D203_PRED corresponds to a base angle and may have a delta angle of 0. Other directionality may be expressed by a base angle and a delta angle. The value of the information regarding the delta angle may fall within the range of {-3, -2, -1, 1, 2, 3}. The physical difference of the delta angle for each base angle may be uniform or non-uniform, and may be in integer or real units.

[0094] In addition to the directionality shown in FIG. 4, other directionality modes (hereinafter referred to as wide-angle directionality modes) may be supported, and the angle of such wide-angle directionality modes may be an angle obtained by inverting some angles shown in FIG. 4 by 180 degrees. At this time, some angles for the wide-angle directionality modes may be determined based on the aspect ratio of the current block.

[0095] Reference Sample Determination Step

[0096] A reference sample for intra prediction of the current block can be determined based on the directionality of the intra prediction mode for the current block. The step of determining the reference sample may include at least one of (1) a step of checking whether a reference sample is available, (2) a step of determining the color space of the current block, (3) a step of signaling a reference line index, or (4) a step of performing filtering on the reference line.

[0097] To determine whether a reference sample is available, you can check whether a reference sample belonging to the left and / or top surrounding area of ​​the current block is available.

[0098] A reference line for intra-prediction of the current block can be determined based on a Multiple Reference Line (MRL) index signaled from the bitstream. The MRL index can specify at least one of a predefined set of multiple reference lines. For example, if the value of the MRL index is 0, a reference line adjacent to the current block can be used. If the value of the MRL index is not 0, a reference line not adjacent to the current block can be used. For example, the number of predefined multiple reference lines may be 4. However, this is merely an example, and all or some of the sample lines located within an N-sample distance from the boundary of the current block may be defined as multiple reference lines, and an MRL index specifying at least one of the multiple reference lines may be signaled.

[0099] The MRL index can be signaled only when the color space of the current block is a luminance component. When the color space of the current block is a chrominance component, the signaling of the MRL index can be omitted, and the value of the MRL index can be derived to 0.

[0100] If the reference line used for intra prediction of the current block is not an adjacent reference line to the current block, additional filtering may be performed after interpolation filtering is performed on that reference line. In this case, the filtering type applied to the non-adjacent reference line may differ from the filtering type applied to the adjacent reference line.

[0101] Filtering performed on a reference line may be at least one of corner filtering or smoothing filtering. Whether corner filtering or smoothing filtering is performed may be determined by a filtering condition. The filtering condition may include at least one of whether an edge filter can be performed, an MRL index, or the directionality of an intra prediction mode.

[0102] For example, if an edge filter is enabled at the sequence level, the value of the MRL index is 0, and the intra prediction mode is not a horizontal mode and / or a vertical mode, at least one of corner filtering or smoothing filtering may be performed on the reference line. Otherwise, IDIF or interpolation filtering described below may be performed on the reference line.

[0103] Corner filtering can be performed based on the weighted sum of the top-left reference sample, the top reference sample, and the left reference sample of the current block. Specifically, corner filtering can be performed as shown in the following Equation 1.

[0104]

[0105] LT CF is the top-left reference sample on which corner filtering has been performed, and L0, LT, and T0 may be the left reference sample, top-left reference sample, and top reference sample of the current block. For example, as shown in FIG. 5, L0 may mean the leftmost reference sample among the top reference samples of the current block. T0 may be the topmost reference sample among the left reference samples of the current block.

[0106] Corner filtering can be performed when at least one of the following conditions is satisfied.

[0107] 1) There is a top reference sample and a left reference sample.

[0108] 2) The sum of the height and width of the current block is 24 or more.

[0109] Whether to apply smoothing filtering and / or the strength of the smoothing filter used for smoothing filtering may be determined based on at least one of the size of the current block or the directionality (or angle) of the intra prediction mode. For example, if the block size is small (e.g., 4x4, 8x4, 4x8) and the angle difference between the intra prediction mode and the vertical / horizontal mode is small (e.g., the angle difference with V_PRED is less than N or the angle difference with H_PRED is less than M, where N and M are integer thresholds), a weak filter may be applied or no filtering may be performed.

[0110] If Intra-Directional Interpolation Filtering (IDIF) is enabled at the sequence level and the intra prediction mode is not vertical or horizontal mode, IDIF can be applied to reference lines. This allows for the generation of reference samples at real-valued positions rather than reference samples at integer positions. The reference samples subjected to IDIF may be samples within reference lines adjacent to the current block, or samples within non-adjacent reference lines.

[0111] Filter coefficients for IDIF can be defined by an agreement between the encoder and decoder as shown in Table 1. In Table 1, A -1 , A0, A1, A2 may be reference samples of integer positions to which IDIF applies. A -1 A0, A1, and A2 may belong to a reference line specified by an MRL index. Subpel in Table 1 may represent a real position defined between two consecutive integer position reference samples.

[0112] SubpelA -1A0A1A200128001-21274-12-31258-23-512313-34-612117-45-911822-56-911627-67-1011232-78-1110937-89-121064 1-810-1210246-911-129852-1012-129456-1013-129061-1114-128566-1115-128171-1216-127676-1217~31Symmetric

[0113] For example, the 12th subpixel (i.e., the reference sample corresponding to the real position of 12 in Table 1) can be derived as shown in the following mathematical equation 2.

[0114]

[0115] A filtering method can be adaptively determined based on the directionality of the intra prediction mode of the current block. For example, referring to FIG. 6, filters with different numbers of taps can be used depending on whether the directionality of the intra prediction mode of the current block is horizontal (e.g., a directionality mode within the delta_h range centered on H_PRED), whether the directionality of the intra prediction mode of the current block is vertical (e.g., a directionality mode within the delta_v range centered on V_PRED), or whether it is neither horizontal nor vertical.

[0116] For directions close to vertical and / or horizontal modes, filters with a smaller number of taps than other directions may be used. For example, a 6-tab filter may be used for directions close to vertical and / or horizontal modes, and an 8-tab filter may be used for other directions.

[0117] delta_v may be a threshold for defining a directional mode with vertical orientation, and delta_h may be a threshold for defining a directional mode with horizontal orientation. delta_v and delta_h may have different values ​​depending on the height and / or width of the current block. For example, delta_v may have a larger value when the width of the current block is greater than its height than delta_v when the width and height of the current block are equal. Additionally, delta_v and delta_h may be determined by the aspect ratio of the current block.

[0118] The interpolation filter can be determined based on the frequency information of a reference sample. The frequency information may include frequency components, the correlation of frequency components between samples, or threshold values. The location of the reference sample may be within a line adjacent to the current prediction block, or within a line not adjacent to the current prediction sample. Alternatively, the reference sample may be within a line where intra-prediction for the current prediction block has already been performed.

[0119] A transformation can be performed on the left or top reference line. Frequency components can be derived by performing a transformation on the reference line. At least one of DCT2 or DST7 can be used as the transformation method. The transformation method can be determined based on at least one of the directionality of the prediction mode or the size of the current block.

[0120] The frequency component may include multiple high-frequency components. By comparing the proportion of the high-frequency component across the entire frequency range with a threshold value, if it is lower than the threshold value, the interpolation filter may be selected as a low-pass filter, and if it is higher than the threshold value, the interpolation filter may be selected as a high-pass filter. The threshold value may be a single value or multiple values. If there are multiple threshold values, multiple intervals based on the multiple threshold values ​​may be established to select different filters for each interval. The threshold value may be determined based on at least one of the size of the current block or quantization parameters. Alternatively, the threshold value may be derived when mapping the wide-angle mode during the prediction mode and directionality determination process. This may be performed when using at least one of the left or top reference lines.

[0121] An interpolation filter can be selected by statistically analyzing the correlation between the left reference sample or the top reference sample. If the correlation is higher than a threshold, a high-pass filter can be selected. If the correlation is lower than a threshold, a low-pass filter can be selected. In this case, the correlation can be calculated by utilizing not only reference samples included in the same reference line, but also additional reference samples within separate reference lines. Information regarding the reference lines available for calculating the correlation can be predefined by an agreement between the encoder and decoder, or determined based on at least one of the directionality of the prediction mode or the size of the current block. For example, the correlation can be derived based on four samples within the left reference line and four samples within the top reference line.

[0122] In both cases, the high-pass filter can be a cubic filter or a DCT interpolation filter. The low-pass filter can be a Gaussian filter or a smoothing filter.

[0123] When performing intra-bilinear prediction on the current block, different interpolation filters may be applied to the forward reference samples and the reverse reference samples, respectively. For example, a 4-tap interpolation filter may be applied to the forward reference samples, and a 2-tap bilinear interpolation filter may be applied to the reverse reference samples. Here, the forward direction may refer to a direction corresponding to the directionality of the intra-prediction mode of the current block, and the reverse direction may refer to a direction opposite to the directionality of the intra-prediction mode. Conversely, a filter with fewer taps may be applied to the forward reference samples than to the reverse reference samples.

[0124] For at least one of the top reference sample or the left reference sample, asymmetric interpolation filtering may be performed based on the available buffer of the reference sample. The available buffer of the reference sample may be a sample existing on the same line as the reference line. When performing asymmetric interpolation filtering, non-linear filtering may be performed. When performing non-linear filtering, samples outside the same line (samples that are not in the available buffer) may not be referenced.

[0125] Intra-prediction execution step

[0126] Intra prediction can be performed for the current block based on a reference sample constructed through the aforementioned method.

[0127] For example, the intra prediction of the current block may be performed based on intra bi-prediction, which shall be referred to as intra bi-prediction. To this end, an Intra Bi-prediction (IBP) flag may be signaled from the bitstream. The IBP flag may indicate whether to perform intra bi-prediction for the current block. If the value of the IBP flag is 1, intra bi-prediction may be performed. If the value of the IBP flag is 0, intra bi-prediction may not be performed.

[0128] FIG. 7 illustrates an intra-bidirectional prediction according to the present disclosure. Referring to FIG. 7, a prediction sample of the current block (a sample at position (x, y) in FIG. 7) can be derived through a weighted sum of reference sample A and reference sample B. Reference sample A may be a reference sample specified by the directionality of the intra-prediction mode of the current block. Reference sample B may be a sample located in the reverse direction of the intra-prediction mode of the current block.

[0129] Intra-bidirectional prediction may be applied only when the intra prediction mode of the current block has a specific directionality. For example, intra-bidirectional prediction may be applied only for at least one of the following cases: when delta angle is 0, when delta angle is +2 and / or -2, when it has a directionality (angle) smaller than V_PRED, or when it has a directionality (angle) larger than H_PRED. That is, the IBP flag may be signaled only when the above conditions are satisfied.

[0130] The weights applied to the reference sample for intra-bidirectional prediction can be adaptively determined based on the distance between the prediction sample and the reference sample within the current block. In this case, if the directionality of the intra-prediction mode is close to the vertical direction, such as in the vertical mode (V_PRED) or D67_PRED, or close to the horizontal direction, such as in the horizontal mode (H_PRED) or D203_PRED, the current block can be divided into multiple zones, and zone-specific weights can be assigned. The types of weights used may vary depending on the block size.

[0131] For example, the current block can be divided into three zones, and different weights can be applied to each zone. In this case, the three zones may be implicitly determined based on at least one of the directionality of the intra prediction mode or the height / width of the current block. If the intra prediction mode of the current block has a directionality adjacent to V_PRED, the current block can be divided into three zones through horizontal division as shown in FIG. 8(a). Alternatively, if the intra prediction mode of the current block has a directionality adjacent to H_PRED, the current block can be divided into three zones through vertical division as shown in FIG. 8(b). Otherwise, the current block can be divided into three zones as shown in FIG. 8(c). In this case, in FIG. 8, A and B may each be W / 4, and C and D may each be H / 4.

[0132] Alternatively, if the intra prediction mode of the current block has a directionality (D67_PRED, D45_PRED, D203_PRED) adjacent to the diagonal directionality, zones 1 and 2 can be defined as a single zone. That is, the current block can be divided into two zones, and different weights can be used for each zone.

[0133] Offset-based Refinement of Intra Prediction (ORIP) may be performed on the prediction samples of the current block. The ORIP may be performed based on an ORIP flag signaled from the bitstream. The ORIP flag may indicate whether to perform offset-based refinement on the current block. If the value of the ORIP flag is 1, offset-based refinement may be performed on the prediction samples of the current block. If the value of the ORIP flag is 0, offset-based refinement may not be performed on the prediction samples of the current block.

[0134] For example, if the ORIP flag is 1, offset-based correction may be performed on all or part of the area within the current block. Here, the part of the area may be an area adjacent to the left and top boundaries of the current block. Offset-based correction may be performed in MxN units (e.g., 4x4 units). The offset for ORIP may be calculated based on reference samples belonging to the surrounding area of ​​the current block. In this case, a fixed offset value may be used depending on at least one of the position of the MxN block or the intra prediction mode of the current block. Alternatively, different offsets may be used depending on the position of the MxN block based on at least one of the intra prediction mode (or directionality) of the current block, the size of the current block, or the aspect ratio of the current block.

[0135] FIG. 9 is a block diagram illustrating an image encoding method according to the present disclosure.

[0136] A video encoding method performed in a video encoding device according to the present disclosure may include a step of determining an intra prediction mode (S900), a step of determining a reference sample (S910), and a step of performing intra prediction (S920). Specific encoding methods for each step are as described with reference to FIG. 3. The video encoding device may encode a syntax element of an intra prediction mode or a reference sample and transmit it through a bitstream.

[0137] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0138] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0139] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

Claims

1. A step of determining the intra prediction mode for the current block; A step of determining a reference sample of the current block above; and A video decoding method comprising the step of performing intra prediction on the current block.

2. In Paragraph 1, The above reference sample determination includes a reference line determination step; and Image decoding method including a reference line filtering step.

3. In Paragraph 2, The above reference line is determined based on at least one of the available reference lines or multi-line indices, and The above-mentioned reference line filtering is an image decoding method comprising at least one of corner filtering or smoothing filtering.

4. In Paragraph 3, An image decoding method in which the above corner filtering is performed based on at least one of the above left reference sample or the above top reference sample.

5. In Paragraph 1, The above intra prediction includes intra directional interpolation filtering (IDIF), and The above IDIF is an image decoding method performed based on at least one of the intra prediction mode or the frequency information of the reference sample.

6. In Paragraph 5, An image decoding method wherein the frequency information of the reference sample comprises at least one of the frequency component of the reference sample or the correlation of frequency components between the reference samples.

7. In Paragraph 1, The above intra prediction includes intra bidirectional prediction, and The above intra-bidirectional prediction is an image decoding method performed based on at least one of the reference sample or weights.

8. In Paragraph 7, The above reference sample includes a reference sample of the left reference line of the current block or a reference sample of the top reference line of the current block, and The above weights are derived based on the distance between a predicted sample and a reference sample, in an image decoding method.

9. In Paragraph 8, The above weight is a video decoding method in which the current block is divided into multiple sections and corrected for each section.

10. In Paragraph 9, The method for dividing the current block above is an image decoding method determined based on the directionality of the intra prediction mode.

11. Step of determining the intra-prediction mode of the current block; A step of determining a reference sample of the current block above; A video encoding method comprising the step of performing intra prediction on the current block.

12. A non-transient computer-readable storage medium for storing a bitstream generated by the image encoding method according to paragraph 11.

13. A method for transmitting a bitstream for transmitting a bitstream generated by the video encoding method according to paragraph 11.

Images