Image encoding / decoding method and apparatus

By using multiple intra prediction modes and histogram information of reference areas during image decoding/encoding, the problem of low compression efficiency in high-resolution image transmission and storage is solved, and more efficient image encoding is achieved and decode.

CN118975234APending Publication Date: 2024-11-15KT CORP
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202380028146.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-03-14
Filing Date
2023-03-15
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

In the transmission and storage of high-resolution and high-quality images, it is difficult for the prior art to effectively compress and decode image signals, resulting in increased transmission and storage costs.

Method used

By determining and using multiple intra prediction modes during the image decoding/encoding process, intra prediction is performed based on the histogram and gradient information of the reference area, coding efficiency and prediction accuracy are improved.

Benefits of technology

The encoding efficiency and prediction accuracy of the intra prediction mode are improved, and the image transmission and storage costs are reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118975234A_ABST
    Figure CN118975234A_ABST
Patent Text Reader

Abstract

An image encoding / decoding method according to the present invention comprises the steps of: determining at least one intra prediction mode for a current block; and performing intra prediction on the current block based on the at least one intra prediction mode. Here, at least one intra prediction mode may be determined by a histogram derived from a reference region of the current block.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to methods and apparatus for processing image signals. Background Art

[0002] Recently, the demand for high-resolution and high-quality images such as HD (high-definition) images and UHD (ultra-high-definition) images has increased in various application fields. Since the amount of data relatively increases compared with existing image data as image data becomes high-resolution and high-quality, when the image data is transmitted by using a medium such as an existing wired and wireless broadband circuit or when the image data is stored by using an existing storage medium, the transmission cost and the storage cost increase. These problems generated by the image data becoming high-resolution and high-quality can be solved by using an efficient image compression technology.

[0003] There are various technologies, such as inter-frame prediction technology that uses image compression technology to predict pixel values ​​included in a current picture based on previous or subsequent pictures of the current picture, intra-frame prediction technology that predicts pixel values ​​included in a current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequencies and assigns long symbols to values ​​with low occurrence frequencies, etc., and image data can be effectively compressed and transmitted or stored by using these image compression technologies.

[0004] On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service also increases. Video compression technology for efficiently providing high-resolution and ultra-high-resolution stereoscopic image content has been discussed. Summary of the invention

[0005] Technical issues

[0006] An object of the present disclosure is to provide an intra-frame prediction method and device for encoding / decoding image signals.

[0007] An object of the present disclosure is to provide a method and apparatus for determining an intra prediction mode for intra prediction at a decoding side when encoding / decoding an image signal.

[0008] An object of the present disclosure is to provide a method and apparatus for obtaining a prediction block by using a plurality of intra prediction modes when encoding / decoding an image signal.

[0009] The technical effects of the present disclosure may not be limited to the above-mentioned technical effects, and ordinary technicians in the technical field to which the present disclosure belongs may clearly understand other unmentioned technical effects from the following description.

[0010] Technical Solutions

[0011] An image decoding method according to the present disclosure may include: determining at least one intra prediction mode for a current block; and performing intra prediction on the current block based on the at least one intra prediction mode.

[0012] In this case, at least one intra prediction mode may be determined by a histogram derived from a reference region of the current block.

[0013] An image encoding method according to the present disclosure may include: determining at least one intra prediction mode for a current block; and performing intra prediction on the current block based on the at least one intra prediction mode.

[0014] In this case, at least one intra prediction mode may be determined by a histogram derived from a reference region of the current block.

[0015] In the image decoding / encoding method according to the present disclosure, the reference area includes a top reference area at the top position of the current block and a left reference area at the left position of the current block, and a histogram can be generated based on the intra-frame prediction mode and amplitude of each reference sample belonging to the reference area.

[0016] In the image decoding / encoding method according to the present disclosure, the intra prediction mode of the reference sample may be obtained based on the ratio between the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the reference sample.

[0017] In the image decoding / encoding method according to the present disclosure, the amplitude of the reference sample may be derived as the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the reference sample.

[0018] In the image decoding / encoding method according to the present disclosure, the horizontal gradient and the vertical gradient of the reference sample are obtained by applying a filter to the reference sample, and the filter may be a Sobel mask or a Prewitt mask.

[0019] In the image decoding / encoding method according to the present disclosure, a histogram may be obtained by accumulating the amplitudes of reference samples of each intra prediction mode.

[0020] In the image decoding / encoding method according to the present disclosure, at least one intra prediction mode may be determined in descending order of amplitude values ​​accumulated in a histogram.

[0021] In the image decoding / encoding method according to the contents of the present disclosure, when multiple intra-frame prediction modes are selected from a histogram, intra-frame prediction can be performed based on each of the multiple intra-frame prediction modes to obtain multiple prediction blocks, and the final prediction block of the current block can be obtained by performing an averaging operation or a weighted sum operation on the multiple prediction blocks.

[0022] In the image decoding / encoding method according to the present disclosure, intra-frame prediction can be performed based on each of at least one intra-frame prediction mode and a default mode to obtain multiple prediction blocks, and the final prediction block of the current block can be obtained by performing an averaging operation or a weighted sum operation on the multiple prediction blocks.

[0023] In the image decoding / encoding method according to the present disclosure, the default mode may include at least one of a planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0024] There may be provided a computer-readable recording medium storing a bit stream encoded by the image encoding method according to the present disclosure.

[0025] Technical Effects

[0026] According to the present disclosure, the encoding efficiency of the intra prediction mode of the current block may be improved by deriving the intra prediction mode from the decoder side.

[0027] According to the present disclosure, the accuracy of intra prediction can be improved by using multiple intra prediction modes.

[0028] Effects obtainable from the present disclosure are not limited to the above-mentioned effects, and other unmentioned effects may be clearly understood from the following description by a person of ordinary skill in the technical field to which the present disclosure pertains. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

[0030] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0031] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0032] Figure 4 and Figure 5 Examples of multiple intra prediction modes according to the present disclosure are shown.

[0033] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0034] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0035] Figure 8A directional mode based intra prediction method according to the present disclosure is shown.

[0036] Fig. 9 A method for deriving samples at fractional positions is shown.

[0037] Fig.10 and Fig.11 It is shown that for each intra prediction mode, the tangent value of the angle is scaled by a factor of 32.

[0038] Fig.12 and Fig.13 is a diagram for describing an example of setting a reference area.

[0039] Fig.14 The filter coefficients of a Sobel mask or a Prewitt mask are shown.

[0040] Fig.15 The locations where the vertical and horizontal gradients within the reference region are obtained are shown.

[0041] Fig.16 An example is shown in which directional modes are grouped into multiple intra prediction mode groups.

[0042] Fig.17 is a diagram for describing a method for determining an intra prediction mode of a chrominance block based on characteristics of a luminance block.

[0043] Fig.18 is a diagram for describing an example in which an intra prediction mode is determined in units of subblocks. DETAILED DESCRIPTION

[0044] Since the present disclosure can be varied and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood that the present disclosure includes all variations, equivalents, or substitutes included in the concept and technical scope of the present disclosure. When describing each of the drawings, similar reference numerals are used for similar parts.

[0045] Terms such as first, second, etc. may be used to describe various components, but components should not be limited by the terms. Terms are used only to distinguish one component from other components. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component, without exceeding the scope of the rights of the present disclosure. Terms and / or combinations of multiple relative input items or any items of multiple relative input items.

[0046] When a component is referred to as being "linked" or "connected" to another component, it should be understood that the component may be directly linked or connected to the other component, but other components may exist in between. On the other hand, when a component is referred to as being "directly linked" or "directly connected" to another component, it should be understood that other components do not exist in between.

[0047] Because the terms used in this application are only used to describe specific embodiments, they are not intended to limit the present disclosure. Singular expressions include plural expressions unless they clearly have different meanings in the context. In this application, it should be understood that terms such as "including" or "having" refer to the existence of characteristics, numbers, steps, movements, parts, parts or combinations thereof entered in the specification, but do not exclude the possibility of adding or existing in advance one or more other characteristics, numbers, steps, movements, parts, parts or combinations thereof.

[0048] Hereinafter, with reference to the accompanying drawings, a desired embodiment of the present disclosure will be described in more detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated description of the same components is omitted.

[0049] Figure 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present disclosure.

[0050] Reference Figure 1 The image encoding device (100) may include: a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a rearrangement unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150) and a memory (155).

[0051] because Figure 1 Each construction unit shown in is independently shown to represent different characteristic functions in the image encoding device, which does not mean that each construction unit is composed of separate hardware or one software unit. That is, since each construction unit is included by being listed as each construction unit for the convenience of description, at least two construction units of each construction unit may be combined to constitute one construction unit, or one construction unit may be divided into a plurality of construction units to perform functions, and even integrated implementations and separate implementations of each construction unit are included in the scope of the rights of the present disclosure as long as they do not depart from the essence of the present disclosure.

[0052] In addition, some components may be only optional components for improving performance, rather than essential components for performing the basic functions in the present disclosure. The present disclosure may be implemented by including only structural units necessary for realizing the essence of the present disclosure without including components only for improving performance, and a structure including only essential components without including optional components only for improving performance is also included in the scope of the rights of the present disclosure.

[0053] The picture segmentation unit (110) may segment an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a decoding unit (CU). In the picture segmentation unit (110), a picture may be segmented into a combination of a plurality of decoding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of a decoding unit, a prediction unit, and a transform unit according to a predetermined criterion (e.g., a cost function).

[0054] For example, one picture may be divided into a plurality of decoding units. In order to divide the decoding units in the picture, a recursive tree structure such as a quadtree structure may be used, and a decoding unit divided into other decoding units by using one image or a maximum decoding unit as a path may be divided with as many child nodes as the number of divided decoding units. A decoding unit that is no longer divided according to a specific restriction becomes a leaf node. In other words, when it is assumed that only square division is possible for one decoding unit, one decoding unit may be divided into up to four other decoding units.

[0055] Hereinafter, in the embodiments of the present disclosure, a decoding unit may be used as a unit for encoding, or may be used as a unit for decoding.

[0056] The prediction unit may be partitioned in at least one square or rectangular shape, etc., with the same size in one decoding unit, or may be partitioned such that any one of the prediction units partitioned in one decoding unit may have a different shape and / or size from another prediction unit.

[0057] When a prediction unit on which intra prediction is performed based on a coding unit is generated and the prediction unit is not a minimum coding unit, intra prediction may be performed without splitting the prediction unit into a plurality of prediction units N×N.

[0058] The prediction unit (120, 125) may include an inter-frame prediction unit (120) that performs inter-frame prediction and an intra-frame prediction unit (125) that performs intra-frame prediction. It may be determined whether to perform inter-frame prediction or intra-frame prediction on the decoding unit, and detailed information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and the specific content. For example, the prediction method, prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and may be sent to the decoding device. When a specific encoding mode is used, the original block may be encoded as is without generating a prediction block through the prediction unit (120, 125) and sent to the decoding unit.

[0059] The inter-frame prediction unit (120) can predict the prediction unit based on information about at least one picture in the previous picture or the subsequent picture of the current picture, or in some cases can predict the prediction unit based on information about some coding areas in the current picture. The inter-frame prediction unit (120) can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0060] The reference picture interpolation unit may receive reference picture information from a memory (155) and generate pixel information equal to or less than an integer pixel in the reference picture. For luma pixels, an 8-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 4 pixels. For chrominance signals, a 4-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 8 pixels.

[0061] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (block matching algorithm based on full search), TSS (three-step search), NTS (new three-step search algorithm), etc. may be used as methods for calculating motion vectors. Based on the interpolated pixels, the motion vector may have a motion vector value of 1 / 2 or 1 / 4 pixel units. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc. may be used as motion prediction methods.

[0062] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When the neighboring block in the current prediction unit is a block for performing inter-frame prediction and therefore the reference pixel is a pixel for performing inter-frame prediction, the reference pixel included in the block for performing inter-frame prediction can be used by replacing it with the reference pixel information of the surrounding blocks for performing intra-frame prediction. In other words, when the reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one reference pixel among the available reference pixels.

[0063] The prediction mode of the intra prediction may have a directional prediction mode that uses reference pixel information according to a prediction direction when performing prediction and a non-directional mode that does not use directional information. The mode for predicting luminance information may be different from the mode for predicting chrominance information, and the intra prediction mode information for predicting luminance information or the predicted luminance signal information may be used to predict chrominance information.

[0064] When the size of the prediction unit is the same as the size of the transform unit when performing intra prediction, intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit. However, if the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0065] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of AIS filter applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode in the current prediction unit may be predicted based on the intra prediction modes in the prediction units surrounding the current prediction unit. When predicting the prediction mode in the current prediction unit by using the mode information predicted according to the surrounding prediction units, if the intra prediction mode in the current prediction unit is the same as the intra prediction mode in the surrounding prediction units, information that the prediction mode in the current prediction unit is the same as the prediction mode in the surrounding prediction units may be sent by using predetermined flag information, and if the prediction mode in the current prediction unit is different from the prediction mode in the surrounding prediction units, the prediction mode information of the current block may be encoded by performing entropy coding.

[0066] In addition, a residual block including information on a residual value that is a difference value between a prediction unit predicted based on the prediction unit generated in the prediction unit (120, 125) and an original block in the prediction unit may be generated. The generated residual block may be input to the transform unit (130).

[0067] The transform unit (130) can transform the original block and the residual block described below by using a transform method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT, and the residual block includes residual value information in the prediction unit generated by the prediction unit (120, 125). Whether to apply DCT, DST or KLT to transform the residual block can be determined based on intra-frame prediction mode information in the prediction unit used to generate the residual block.

[0068] The quantization unit (135) may quantize the value transformed into the frequency domain in the transform unit (130). The quantization coefficient may be changed according to the importance of the block or image. The value calculated in the quantization unit (135) may be provided to the inverse quantization unit (140) and the rearrangement unit (160).

[0069] The rearrangement unit (160) may perform rearrangement of coefficient values ​​on the quantized residual value.

[0070] The rearrangement unit (160) can change the coefficients in the shape of a two-dimensional block into the shape of a one-dimensional vector by a coefficient scanning method. For example, the rearrangement unit (160) can scan the DC coefficient to the coefficient in the high frequency domain by using a zigzag scanning method and change it into the shape of a one-dimensional vector. Depending on the size of the transform unit and the intra-frame prediction mode, instead of the zigzag scanning, vertical scanning that scans the coefficients in the shape of a two-dimensional block in the column direction or horizontal scanning that scans the coefficients in the shape of a two-dimensional block in the row direction can be used. In other words, which scanning method to use among the zigzag scanning, the vertical direction scanning, and the horizontal direction scanning can be determined according to the size of the transform unit and the intra-frame prediction mode.

[0071] The entropy coding unit (165) may perform entropy coding based on the value calculated by the rearrangement unit (160). For example, the entropy coding may use various coding methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding).

[0072] The entropy coding unit (165) can encode various information from the rearrangement unit (160) and the prediction unit (120, 125), such as residual value coefficient information and block type information in the decoding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0073] The entropy encoding unit (165) may perform entropy encoding on the coefficient values ​​in the decoding unit input from the rearrangement unit (160).

[0074] The inverse quantization unit (140) and the inverse transform unit (145) inversely quantize the value quantized in the quantization unit (135) and inversely transform the value transformed in the transform unit (130). The residual value generated by the inverse quantization unit (140) and the inverse transform unit (145) can be combined with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra-frame prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0075] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0076] The deblocking filter can remove block distortion generated by the boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it can be determined whether the deblocking filter will be applied to the current block based on the pixels included in several rows or columns included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter can be applied according to the required deblocking filter strength. In addition, in the case of applying the deblocking filter, when horizontal filtering and vertical filtering are performed, horizontal direction filtering and vertical direction filtering can be set to parallel processing.

[0077] The offset correction unit can correct the offset from the original image in units of pixels for the image to be deblocked. In order to perform offset correction on a specific picture, the area where the offset is to be performed can be determined after dividing the pixels included in the image into a certain number of areas, and a method of applying the offset to the corresponding area or a method of applying the offset by considering edge information of each pixel can be used.

[0078] Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing a filtered reconstructed image with an original image. After pixels included in an image are divided into predetermined groups, filtering may be performed differently on each group by determining a filter to be applied to the corresponding group. Information on whether ALF is applied may be transmitted per coding unit (CU) for a luminance signal, and the shape and filter coefficient of the ALF filter to be applied may differ for each block. In addition, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to be applied.

[0079] The memory (155) may store a reconstructed block or picture calculated by the filter unit (150), and when inter-frame prediction is performed, the stored reconstructed block or picture may be provided to the prediction unit (120, 125).

[0080] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0081] Reference Figure 2 , the image decoding device (200) may include: an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transform unit (225), a prediction unit (230, 235), a filter unit (240) and a memory (245).

[0082] When an image bitstream is input from the image encoding device, the input bitstream may be decoded according to a process reverse to that of the image encoding device.

[0083] The entropy decoding unit (210) can perform entropy decoding according to a process opposite to the process of performing entropy encoding in the entropy encoding unit of the image encoding device. For example, various methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding) and CABAC (Context Adaptive Binary Arithmetic Coding) can be applied in response to the method performed in the image encoding device.

[0084] The entropy decoding unit (210) can decode information related to intra-frame prediction and inter-frame prediction performed in the encoding device.

[0085] The rearrangement unit (215) may perform rearrangement based on a method of rearranging the bit stream entropy decoded in the entropy decoding unit (210) in the coding unit. The coefficients expressed in the one-dimensional vector form may be rearranged by reconfiguring the coefficients in the two-dimensional block form. The rearrangement unit (215) may receive information related to coefficient scanning performed in the coding unit, and perform rearrangement by performing scanning inversely based on a scanning order performed in the corresponding coding unit.

[0086] The inverse quantization unit (220) may perform inverse quantization based on the quantization parameter provided from the encoding device and the coefficient value of the rearranged block.

[0087] The inverse transform unit (225) can perform the transform performed in the transform unit, that is, the inverse transform for DCT, DST and KLT (that is, inverse DCT, inverse DST and inverse KLT) on the result of the quantization performed in the image encoding device. The inverse transform can be performed based on the transmission unit determined in the image encoding device. In the inverse transform unit (225) of the image decoding device, the transform technology (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of information such as a prediction method, a size of a current block, a prediction mode, etc.

[0088] The prediction unit (230, 235) may generate a prediction block based on information related to generation of the prediction block provided from the entropy decoding unit (210) and pre-decoded block or picture information provided from the memory (245).

[0089] As described above, when intra prediction is performed in the same manner as the operation in the image encoding device, when the size of the prediction unit is the same as the size of the transform unit, the intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit, but when the size of the prediction unit is different from the size of the transform unit when performing intra prediction, the intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0090] The prediction unit (230, 235) may include: a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information input from the entropy decoding unit (210), such as prediction unit information, prediction mode information of an intra-frame prediction method, information related to motion prediction of an inter-frame prediction method, etc., divide the prediction unit in the current decoding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit (230) may perform inter-frame prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of a current picture including the current prediction unit, by using information required for inter-frame prediction in the current prediction unit provided from the image encoding device. Alternatively, inter-frame prediction may be performed based on information about some areas pre-reconstructed in the current picture including the current prediction unit.

[0091] In order to perform inter prediction, whether a motion prediction method in a prediction unit included in a corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or an intra block copy mode may be determined on a coding unit basis.

[0092] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that performs intra prediction, intra prediction can be performed based on intra prediction mode information in the prediction unit provided from the image encoding device. The intra prediction unit (235) may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As a part of performing filtering on the reference pixels of the current block, the AIS filter can be applied by determining whether to apply the filter according to the prediction mode in the current prediction unit. By using the prediction mode and AIS filter information in the prediction unit provided from the image encoding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0093] When the prediction mode in the prediction unit is a prediction unit that performs intra-frame prediction based on a pixel value interpolated from a reference pixel, the reference pixel interpolation unit may interpolate the reference pixel so as to generate the reference pixel in units of pixels equal to or less than an integer value. When the prediction mode in the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0094] The reconstructed block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0095] Information about whether a deblocking filter is applied to a corresponding block or picture and information about whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the image encoding device. Information related to the deblocking filter provided from the image encoding device may be provided in the deblocking filter of the image decoding device, and deblocking filtering of the corresponding block may be performed in the image decoding device.

[0096] The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image when encoding is performed, and the offset value information.

[0097] ALF may be applied to the decoding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoding device. Such ALF information may be provided by being included in a specific parameter set.

[0098] The memory (245) may store the reconstructed picture or block to be used as a reference picture or a reference block, and provide the reconstructed picture to the output unit.

[0099] As described above, hereinafter, in the embodiments of the present disclosure, for convenience of description, a decoding unit is used as a term of a decoding unit, but may be a unit that performs decoding as well as encoding.

[0100] In addition, since the current block represents a block to be encoded / decoded, according to the encoding / decoding step, the current block may represent a coding tree block (or coding tree unit), a decoding block (or decoding unit), a transform block (or transform unit), or a prediction block (or prediction unit), etc. In the present specification, a "unit" may represent a basic unit for performing a specific encoding / decoding process, and a "block" may represent a pixel array of a predetermined size. Unless otherwise classified, "block" and "unit" may be used interchangeably. For example, in the embodiments described later, it may be understood that a coding block (coding block) and a decoding unit (decoding unit) may be used interchangeably.

[0101] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0102] Reference Figure 3 , a reference line for intra prediction of a current block may be determined ( S300 ).

[0103] The current block can use one or more of the multiple reference line candidates predefined in the image encoding / decoding device as a reference line for intra-frame prediction. Here, the multiple predefined reference line candidates may include: adjacent reference lines adjacent to the current block to be decoded and N non-adjacent reference lines that are 1 sample to N samples away from the boundary of the current block. N can be an integer of 1, 2, 3 or more. Hereinafter, for ease of description, it is assumed that the multiple reference line candidates available for the current block consist of adjacent reference line candidates and three non-adjacent reference line candidates, but is not limited to this. In other words, of course, the multiple reference line candidates available for the current block may include four or more non-adjacent reference line candidates.

[0104] The image encoding device may determine the best reference line candidate among multiple reference line candidates and encode an index for specifying the best reference line candidate. The image decoding device may determine the reference line of the current block based on the index signaled by the bitstream. The index may specify any one of the multiple reference line candidates. The reference line candidate specified by the index may be used as the reference line of the current block.

[0105] The number of indexes signaled to determine the reference line of the current block may be 1, 2, or more. As an example, when the number of indexes signaled above is 1, the current block may perform intra-frame prediction by using only a single reference line candidate specified by the index signaled among the multiple reference line candidates. Alternatively, when the number of indexes signaled above is at least 2, the current block may perform intra-frame prediction by using multiple reference line candidates specified by multiple indexes among the multiple reference line candidates.

[0106] Reference Figure 3 , the intra prediction mode of the current block can be determined (S310).

[0107] The intra prediction mode of the current block can be determined among a plurality of intra prediction modes predefined in the image encoding / decoding device. Figure 4 and Figure 5 A number of predefined intra prediction modes are described.

[0108] Figure 4 Examples of multiple intra prediction modes according to the present disclosure are shown.

[0109] Reference Figure 4, the multiple intra-frame prediction modes predefined in the image encoding / decoding device may consist of a non-directional mode and a directional mode. The non-directional mode may include at least one of a planar mode and a DC mode. The directional mode may include directional mode 2 to directional mode 66.

[0110] Directional mode can be compared Figure 4 The one shown in is expanded even more. Figure 5 An example in which the directional mode is extended is shown.

[0111] exist Figure 5 , it is shown that modes -1 to -14 and modes 67 to 80 are added. These directional modes may be referred to as wide-angle intra prediction modes. Whether to use the wide-angle intra prediction mode may be determined according to the shape of the current block. As an example, if the current block is a non-square block whose width is greater than the height, some directional modes (e.g., No. 2 to No. 15) may be converted into wide-angle intra prediction modes between No. 67 and No. 80. On the other hand, if the current block is a non-square block whose height is greater than the width, some directional modes (e.g., No. 53 to No. 66) may be converted into wide-angle intra prediction modes between No. -1 and No. -14.

[0112] The range of available wide-angle intra prediction modes may be adaptively determined according to the aspect ratio of the current block. Table 1 shows the range of available wide-angle intra prediction modes according to the aspect ratio of the current block.

[0113] [Table 1]

[0114] Width / Height Available wide-angle intra prediction mode range Width / Height = 16 67 ~80 Width / Height = 8 67 ~78 Width / Height = 4 67 ~76 Width / Height = 2 67 ~74 Width / Height = 1 none Width / Height = 1 / 2 -1 ~-8 Width / Height = 1 / 4 -1 ~-10 Width / Height = 1 / 8 -1 ~-12 Width / Height = 1 / 16 -1 ~-14

[0115] Among multiple intra prediction modes, K candidate modes (most probable modes MPM) may be selected. A candidate list including the selected candidate modes may be generated. An index indicating any one of the candidate modes belonging to the candidate list may be notified by a signal. The intra prediction mode of the current block may be determined based on the candidate mode indicated by the index. As an example, the candidate mode indicated by the index may be configured as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be determined based on the value of the candidate mode indicated by the index and a predetermined difference value. The difference value may be defined as the difference between the value of the intra prediction mode of the current block and the value of the candidate mode indicated by the index. The difference value may be notified by a signal through a bitstream. Alternatively, the difference value may be a value predefined in an image encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag indicating whether there is a mode identical to the intra prediction mode of the current block in the candidate list. As an example, when the flag is a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of the multiple candidate modes belonging to the candidate list may be notified by a signal. The candidate mode indicated by the index can be configured as the intra-frame prediction mode of the current block. On the other hand, when the flag is a second value, any one of the remaining intra-frame prediction modes can be configured as the intra-frame prediction mode of the current block. The remaining intra-frame prediction modes may refer to a mode other than the candidate modes belonging to the candidate list among multiple predefined intra-frame prediction modes. When the flag is a second value, an index indicating any one of the remaining intra-frame prediction modes can be signaled. The intra-frame prediction mode indicated by the signaled index can be configured as the intra-frame prediction mode of the current block. The intra-frame prediction mode of a chroma block can be selected from the intra-frame prediction mode candidates of multiple chroma blocks. To this end, the index information indicating one of the intra-frame prediction mode candidates of the chroma block can be explicitly encoded and the index information can be signaled through the bitstream. Table 2 shows the intra-frame prediction mode candidates for the chroma block.

[0116] [Table 2]

[0117] index Intra prediction mode candidates for chroma blocks 0 0 (plane) 1 50(vertical) 2 18(Horizontal) 3 1(DC) 4 DM

[0118] In the example of Table 2, direct mode (DM) means setting the intra prediction mode of the luminance block existing at the same position as the chrominance block to the intra prediction mode of the chrominance block. Therefore, the intra prediction mode of the chrominance block can also be set to Figure 4 or Figure 5 The intra prediction mode of the current block may be used to determine the reference line of the current block, and in this case, S310 may be performed before S300. Figure 3 , intra prediction may be performed on the current block based on the reference line and the intra prediction mode of the current block (S320).

[0119] In the following, reference will be made to Figures 6 to 8 The intra prediction method of each intra prediction mode is described in detail. However, for ease of description, it is assumed that a single reference line is used for intra prediction of the current block, but even in the case of using multiple reference lines, the intra prediction method described later can be applied in the same / similar manner.

[0120] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0121] Reference Figure 6 , T represents the reference sample located at the upper right corner of the current block, and L represents the reference sample located at the lower left corner of the current block. P1 may be generated by interpolation in the horizontal direction. As an example, P1 may be generated by interpolating T using a reference sample located on the same horizontal line as P1. P2 may be generated by interpolation in the vertical direction. As an example, P2 may be generated by interpolating L using a reference sample located on the same vertical line as P2. The current sample in the current block may be predicted by a weighted sum of P1 and P2, as shown in Equation 1 below.

[0122] [Equation 1]

[0123] (α×P1+β×P2) / (α+β)

[0124] In Equation 1, the weight α and the weight β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, the weight α and the weight β may have the same value or different values. If the width and height of the current block are the same, the weight α and the weight β may be configured to be the same, and the predicted sample of the current sample may be configured as the average of P1 and P2. In the case where the width and height of the current block are not the same, the weight α and the weight β may have different values. As an example, when the width is greater than the height, a smaller value may be configured as a weight corresponding to the width of the current block, and a larger value may be configured as a weight corresponding to the height of the current block. Conversely, when the width is greater than the height, a larger value may be configured as a weight corresponding to the width of the current block, and a smaller value may be configured as a weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block may be referred to as β, and the weight corresponding to the height of the current block may be referred to as α.

[0125] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0126] See also Figure 7, the average value of the surrounding samples adjacent to the current block can be calculated, and the calculated average value can be configured as the prediction value of all samples in the current block. Here, the surrounding samples may include the top reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value may be calculated by using only the top reference sample or the left reference sample. As an example, in the case where the width of the current block is greater than the height, the average value may be calculated using only the top reference sample of the current block. Alternatively, in the case where the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated using only the top reference sample of the current block. Alternatively, in the case where the ratio of the width and height of the current block is less than or equal to a predetermined threshold, the average value may be calculated using only the top reference sample of the current block. On the other hand, in the case where the width of the current block is less than the height, the average value may be calculated using only the left reference sample of the current block. Alternatively, in the case where the ratio of the width and height of the current block is less than or equal to a predetermined threshold, the average value may be calculated using only the left reference sample of the current block. Alternatively, in a case where the ratio of the width and height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated using only left reference samples of the current block.

[0127] Figure 8 A directional mode based intra prediction method according to the present disclosure is shown.

[0128] In the case where the intra-frame prediction mode of the current block is a directional mode, the reference line can be projected according to the angle of the corresponding directional mode. When there is a reference sample at the projection position, the corresponding reference sample can be configured as a prediction sample of the current sample. If there is no reference sample at the projection position, a sample corresponding to the projection position can be generated by using one or more surrounding samples adjacent to the projection position. As an example, interpolation can be performed based on two or more surrounding samples that are bidirectionally adjacent to the projection position to generate a sample corresponding to the projection position. Alternatively, a surrounding sample adjacent to the projection position can be configured as a sample corresponding to the projection position. In this case, among a plurality of surrounding samples adjacent to the projection position, the surrounding sample closest to the projection position can be used. The sample corresponding to the projection position can be configured as a prediction sample of the current sample.

[0129] Reference Figure 8, for the current sample B, when the reference line is projected at the corresponding position according to the angle of the intra-frame prediction mode, there is a reference sample at the projection position (i.e., the reference sample at the integer position R3). In this case, the reference sample at the projection position can be configured as a prediction sample for the current sample B. For the current sample a, when the reference line is projected at the corresponding position according to the angle of the intra-frame prediction mode, there is no reference sample at the projection position (i.e., the reference sample at the integer position). In this case, interpolation can be performed based on surrounding samples (e.g., R2 and R3) adjacent to the projection position to generate a sample (r) at a fractional position. The generated sample (r) at the fractional position can be configured as a prediction sample for the current sample A.

[0130] Fig. 9 A method for deriving samples at fractional positions is shown.

[0131] exist Fig. 9 In the example of , variable h refers to the vertical distance (i.e., vertical distance) from the position of prediction sample A to the reference sample line, and variable w refers to the horizontal distance (i.e., horizontal distance) from the position of prediction sample A to the fractional position sample. In addition, variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and variable x refers to the fractional position.

[0132] The variable w can be obtained according to the following equation 2.

[0133] [Equation 2]

[0134] w=htanθ

[0135] Then, if we remove the integer positions from the variable w, we can finally get the fractional positions.

[0136] The fractional position samples may be generated by interpolating adjacent integer position reference samples. As an example, the fractional position reference samples at the x position may be generated by interpolating integer position reference samples R2 and integer position reference samples R3.

[0137] When deriving fractional position samples, a scaling factor can be used to avoid real number operations. As an example, when the scaling factor f is set to 32, such as Figure 8 For the example shown in (b), the distance between adjacent integer reference samples can be set to 32 instead of 1.

[0138] In addition, the tangent value of the angle θ determined according to the directionality of the intra prediction mode may also be enlarged by using the same scaling factor (eg, 32).

[0139] Fig.10 and Fig.11It is shown that for each intra prediction mode, the tangent value of the angle is scaled by a factor of 32.

[0140] Fig.10 shows the scaling result of the tangent value for non-wide angle intra prediction mode, Fig.11 The scaling results of the tangent values ​​for the Wide-angle intra prediction mode are shown.

[0141] The intra prediction mode of the current block can be derived by using reference samples around the current block. Specifically, after calculating the gradients of the reference samples in the horizontal and vertical directions, the intra prediction mode of the current block can be derived by using the calculated gradients.

[0142] Fig.12 is a diagram for describing an example of setting a reference area.

[0143] For ease of description, it is assumed that the size of the current block is 4×4.

[0144] A reference region for deriving an intra prediction mode for a current block may be set. Fig.12 , it is assumed that a w0 column adjacent to the left side of the current block and an h0 row adjacent to the top of the current block are set as the reference area.

[0145] The number of columns (w0) and / or the number of rows (h0) constituting the reference region may be fixed in the encoder and the decoder. Alternatively, the number of columns (w0) and / or the number of rows (h0) may be determined based on at least one of the following: the size / shape of the current block, whether intra sub-partitioning (ISP) is applied to the current block, or whether the current block is adjacent to a CTU boundary.

[0146] As another example, the size of the reference area can be determined according to the shape of the filter applied to the reference area. Specifically, the horizontal length and vertical length of the filter can be set to the number of columns w0 and the number of rows h0, respectively. As an example, when assuming that the filter described later is used Fig.14 When the 3×3 mask is shown in , the number of columns w0 and the number of rows h0 can be set to 3 respectively.

[0147] The reference area may extend beyond the right and / or bottom boundaries of the current block. Fig.12 In the example shown in , it is shown that the reference area extends w1 from the right boundary of the current block, and extends h1 from the bottom boundary of the current block.

[0148] The right extension distance w1 and / or the bottom extension distance h1 may be set to be the same as the width and / or height of the current block. As an example, if the size of the current block is 4×4, the right extension distance w1 may be set to 4 (the same as the width of the current block), and the bottom extension distance h1 may be set to 4 (the same as the height of the current block).

[0149] As another example, Fig.13 As shown in the example, you can set the reference area.

[0150] Specifically, Fig.13 In the example shown in (a), the right extension distance w1 and / or the bottom extension distance h1 can be set to 0. Fig.13 In the example shown in (b), the top reference region may consist only of reference samples with x-axis coordinates between 0 and (w-1), and the left reference region may consist only of reference samples with y-axis coordinates between 0 and (h-1). Here, w represents the width of the current block, and h represents the height of the current block.

[0151] As another example, a reference line candidate or at least one of the reference line candidates used for intra prediction of the current block may be set as a reference region.

[0152] As another example, depending on whether the current block is adjacent to a CTU boundary, the reference region may consist of only the top reference region, or may consist of only the left reference region.

[0153] Filtering (ie, convolution) may be performed using a mask within the reference region. In this case, the filter used may be at least one of a Sobel mask or a Prewitt mask that outputs gradient values.

[0154] Fig.14 The filter coefficients of a Sobel mask or a Prewitt mask are shown.

[0155] Can be used with Fig.14 , a different type of filter than the one shown in can be applied to the reference region. As an example, a 1D filter of 1×3 or 3×1 size, a rectangular filter of 2×3 or 2×3 size, a cross filter, or a diamond filter can be applied to the reference region instead of a 3×3 size square filter. Alternatively, a filter of the same size as Fig.14 Filters of different filter sizes (eg, 2×2, 4×4, or 5×5, etc.) as shown in FIG. 5 are applied to the reference region.

[0156] The type of filter applied to the reference region may be predefined in the encoder and the decoder. Alternatively, after predefining a plurality of filter candidates, index information indicating one of the plurality of filter candidates may be encoded and explicitly signaled through a bitstream.

[0157] As another example, at least one filter candidate from among the plurality of filter candidates may be adaptively selected based on at least one of the size / shape of the current block, whether an ISP is applied to the current block, or whether the current block is adjacent to a CTU boundary. Here, the adjacent block may include at least one of a top adjacent block of the current block or a left adjacent block of the current block.

[0158] The type of filter applied to the top reference region may be different from the type of filter applied to the left reference region.

[0159] By applying a vertical mask to a specific reference sample in the reference region, the vertical gradient Dy of the reference sample can be obtained. In addition, by applying a horizontal mask to a specific reference sample in the reference region, the horizontal gradient Dx of the reference sample can be obtained.

[0160] Fig.15 The locations where the vertical and horizontal gradients within the reference region are obtained are shown.

[0161] like Fig.14 In the example shown in , when it is assumed that a mask of size 3×3 is applied, for each reference sample in the reference samples that are not adjacent to the boundary of the reference region, a vertical gradient Dy and a horizontal gradient Dx can be obtained. As an example, when w0 and h0 are 3 and w1 and h1 are 4, as Fig.14 In the example shown in , for each of the 17 reference samples, 17 vertical gradients Dy and 17 horizontal gradients Dx can be obtained.

[0162] When applied with Fig.14 When a filter of a different size or a different shape is used than the one shown in , the vertical gradient Dy and the horizontal gradient Dx can be obtained for more / fewer reference samples than shown.

[0163] Based on the vertical gradient Dy and the horizontal gradient Dx of each of the reference samples, an intra prediction mode for each of the reference samples may be determined.

[0164] A method for determining an intra prediction mode of a reference sample by using a vertical gradient Dy and a horizontal gradient Dx will be described.

[0165] As an example, when any one of the vertical gradient Dy and the horizontal gradient Dx is 0, the directional mode of the corresponding reference sample can be determined as the horizontal mode (number 18) or the vertical mode (number 50). Specifically, when the horizontal gradient Dx is 0 and the vertical gradient Dy is not 0, the intra-frame prediction mode of the corresponding reference sample can be determined as the vertical mode (number 50). Conversely, when the vertical gradient Dy is 0 and the horizontal gradient Dx is not 0, the intra-frame prediction mode of the corresponding reference sample can be determined as the horizontal mode (number 18).

[0166] When both the vertical gradient Dy and the horizontal gradient Dx are not 0, one of the remaining directional modes except the horizontal mode and the vertical mode may be determined as the intra prediction mode of the reference sample.

[0167] Here, the intra prediction mode group to which the intra prediction mode of the reference sample belongs may be determined by comparing the absolute value of the vertical gradient Dy with the absolute value of the horizontal gradient Dx. Here, the intra prediction mode group may be composed of a plurality of directional modes having similar directivities.

[0168] Fig.16 An example is shown in which directional modes are grouped into multiple intra prediction mode groups.

[0169] exist Fig.16 , it is shown that the directional modes are classified into four intra prediction mode groups (a to d) based on the horizontal mode (number 18), the diagonal mode (number 34), and the vertical mode (number 50).

[0170] In the example shown, groups a and b have symmetrical structures based on a horizontal pattern (number 18), while groups c and d have symmetrical structures based on a vertical pattern (number 50).

[0171] In addition, the angles of the directional patterns No. 36 to No. 66 are the same as the angles obtained by transposing the angles of the patterns No. 2 to No. 34.

[0172] If the absolute value of the horizontal gradient Dx of the reference sample is greater than the absolute value of the vertical gradient Dy, the directional pattern corresponding to the reference sample may belong to group a or group b.

[0173] On the contrary, if the absolute value of the vertical gradient Dy of the reference sample is greater than the gradient of the horizontal gradient Dx, the directional pattern corresponding to the reference sample may belong to group c or group d.

[0174] Table 3 shows the intra prediction mode group to which the intra prediction mode of the reference sample belongs according to the magnitudes of the horizontal gradient Dx and the vertical gradient Dy.

[0175] [Table 3]

[0176]

[0177] The horizontal gradient Dx and the vertical gradient Dy of the reference sample can be used to derive the gradient of the directional pattern to be assigned to the reference sample. To this end, as shown in the following equation 3, a variable R representing the ratio between the horizontal gradient and the vertical gradient can be derived.

[0178] [Equation 3]

[0179] R=(|Dx|>=|Dy|)? (|Dy| / |Dx|): (|Dx| / |Dy|)

[0180] As shown in Equation 3, the variable R may be obtained by using one of the horizontal gradient Dx and the vertical gradient Dy having a larger absolute value as a denominator.

[0181] Thereafter, the directional pattern of the reference sample may be determined by comparing the variable R with the tangent value (tanθ) of the angle of each directional pattern. Specifically, the directional pattern having the same or most similar tangent value as the variable R may be assigned to the reference sample.

[0182] In this case, if Fig.10 or Fig.11 In the example shown in , if the tangent value of the angle of each directional pattern is stored in a scaled state in the encoder and decoder, the directional pattern of the reference sample can be determined by scaling the variable R with the same scaling factor.

[0183] Next, the amplitude of each reference sample can be derived. As shown in Equation 4 below, the amplitude can be derived as the sum of the absolute value of the horizontal gradient Dx and the absolute value of the vertical gradient Dy.

[0184] [Equation 4]

[0185] amplitude=|Dx|+|Dy|

[0186] Next, for each intra prediction mode, the amplitude value of each reference sample assigned with the same intra prediction mode may be accumulated.

[0187] [Equation 5]

[0188] histogram[intra_mode]+=amplitude

[0189] In Equation 5, intra_mode represents an intra prediction mode. As an example, the accumulated amplitude value of the directional mode with mode number N may be obtained by combining the amplitude values ​​of the reference samples assigned with mode number N in the reference region, and the accumulated amplitude value of the directional mode with mode number M may be obtained by combining the amplitude values ​​of the reference samples assigned with mode number M in the reference region.

[0190] The buffer storing the accumulated amplitude value can be initialized in units of blocks. As an example, when a reference area around a current block is specified, the accumulated amplitude value of each intra prediction mode may be initialized to 0.

[0191] If a histogram recording the accumulated amplitude values ​​of each intra-frame prediction mode is obtained through this process, at least one intra-frame prediction mode can be selected in descending order of the accumulated amplitude values ​​in the histogram. The number of selected intra-frame prediction modes can be M, and M can be a natural number equal to or greater than 1. The value of M can be predefined in the encoder and decoder. Alternatively, the value of M can be adaptively determined by considering the size / shape of the current block or whether the ISP is applied to at least one of the current blocks. In other words, M intra-frame prediction modes can be selected in descending order of the accumulated amplitude values.

[0192] At least one intra prediction mode selected from the histogram may be set as the intra prediction mode of the current block, and a prediction block of the current block is obtained based on the intra prediction mode of the current block. As an example, when an intra prediction mode is selected from the histogram, a prediction block obtained based on the selected intra prediction mode may be used as a final prediction block of the current block.

[0193] When multiple intra prediction modes are selected from the histogram, intra prediction can be performed based on each of the multiple intra prediction modes. Therefore, when multiple prediction blocks are generated, the final prediction block of the current block can be obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

[0194] In this case, for the weighted sum operation, the weight applied to each prediction block may be determined based on the magnitude of the intra-frame prediction mode. In other words, the largest weight may be assigned to the prediction block derived based on the intra-frame prediction mode with the largest magnitude among the multiple intra-frame prediction modes, while the smallest weight may be assigned to the prediction block derived based on the intra-frame prediction mode with the smallest magnitude.

[0195] In this case, the weight assigned to each prediction block may be determined based on the ratio between the amplitudes. Alternatively, after pre-storing the weight value of each amplitude level, the weight mapped to the amplitude level of the corresponding intra prediction mode may be applied to the prediction block.

[0196] The prediction block of the current block may be obtained by further considering at least one default mode and at least one intra prediction mode selected from the histogram. As an example, intra prediction may be performed based on each of the intra prediction mode selected from the histogram and the default mode to obtain multiple prediction blocks of the current block. Thereafter, the final prediction block of the current block may be obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

[0197] The number N of default modes may be 0 or an integer equal to or greater than 1. When M intra prediction modes are selected from the histogram, (M+N) prediction blocks may be obtained by performing intra prediction based on each of the M intra prediction modes and the N default modes. Thereafter, a final prediction block of the current block may be obtained by performing an average operation or a weighted sum operation on the (M+N) prediction blocks.

[0198] The number N of default modes may be predefined in the encoder and decoder. Alternatively, the number N of default modes may be adaptively determined based on at least one of the following: the size / shape of the current block, whether ISP is applied to the current block, or whether at least one intra prediction mode selected from the histogram includes a default mode.

[0199] The default mode may include at least one of a planar mode, a DC mode, or a predefined directional mode.

[0200] The encoder and decoder may be set to use a predefined mode (eg, planar mode) among the modes listed above as a default mode.

[0201] Alternatively, the type of the default mode can be adaptively determined according to the type of the directional mode selected by the histogram. As an example, when at least one directional mode selected by the histogram is a vertical mode or a horizontal mode, the plane mode or the DC mode can be set as the default mode. On the other hand, when the vertical mode and / or the horizontal mode are not selected by the histogram, the vertical mode or the horizontal mode can be set as the default mode.

[0202] According to the shape of the current block, it can be determined whether the wide-angle intra prediction mode is available. As an example, if the current block has a square shape with the same width and height, the directional mode selected from the histogram may be composed of non-wide-angle intra prediction modes. On the other hand, if the current block has a non-square shape with different width and height, some of the directional modes selected from the histogram may be converted into wide-angle intra prediction modes.

[0203] When performing intra prediction based on the intra prediction mode derived from the histogram, a predefined reference line may be used. Here, the predefined reference line may be an adjacent reference line (eg, index 0) or a non-adjacent reference line (eg, index 1) adjacent to the current block.

[0204] The above information indicating whether the method for performing intra prediction by selecting an intra prediction mode from a histogram is applied may be encoded and notified through a bitstream signal. The information may be a 1-bit flag.

[0205] Alternatively, whether to select the intra prediction mode by a histogram may be determined based on at least one of the size / shape of the current block, whether an ISP is applied to the current block, whether the current block is adjacent to a CTU boundary, or whether a neighboring block is encoded by intra prediction.

[0206] As an example, in a case where at least one of a top neighboring block or a left neighboring block of a current block is not encoded through intra prediction, a method for selecting an intra prediction mode through a histogram may be applied to the current block.

[0207] The method for selecting an intra prediction mode by a histogram may be applied to both the luminance component and the chrominance component. Alternatively, the above method may be applied only to the luminance component. Alternatively, it may be determined independently for each of the luminance component and the chrominance component whether to select an intra prediction mode by a histogram.

[0208] When encoding / decoding a chroma block, the intra prediction mode of the chroma block may be determined based on the characteristics of the luminance block. Here, the characteristics of the luminance block may represent the gradient of the samples belonging to the luminance block (specifically, the horizontal gradient Dx and the vertical gradient Dy). In addition, the samples used to derive the gradient may refer to the reconstructed samples within the luminance block.

[0209] Fig.17 is a diagram for describing a method for determining an intra prediction mode of a chrominance block based on characteristics of a luminance block.

[0210] The entire luminance block or a partial area of ​​the luminance block can be set as the reference area. Then, by filtering, the gradient of the samples in the reference area can be obtained. As an example, Fig.17 In the example shown in Fig.14 When a mask of size 3×3 is used as shown in , for each of the remaining samples except samples adjacent to the boundary of the luminance block, a horizontal gradient Dx and a vertical gradient Dy can be obtained.

[0211] As another example, at least a portion of the luminance block and the pre-reconstructed area around the luminance block may be set as the reference area. As an example, the reference area may also include at least one of at least one reconstructed sample line adjacent to the top of the luminance block or at least one reconstructed sample line adjacent to the left side of the luminance block.

[0212] After determining the intra prediction mode of each sample by using the horizontal gradient and the vertical gradient of each sample, a histogram can be derived based on the accumulated amplitude value of each intra prediction mode. Since the method for deriving the histogram has been described in detail by Equations 3 to 5, its detailed description will be omitted.

[0213] The intra prediction mode with the maximum accumulated amplitude value in the histogram can be set as the intra prediction mode of the chrominance block located at the same position as the luminance block. In other words, by performing intra prediction based on the intra prediction mode with the maximum accumulated amplitude value in the histogram, a prediction block of the chrominance block can be obtained.

[0214] As another example, after selecting N intra-frame prediction modes with large accumulated amplitude values ​​from the histogram, intra-frame prediction can be performed based on each of the N intra-frame prediction modes to obtain N prediction blocks of the chrominance block. Here, N can be a natural number equal to or greater than 1. N can be determined based on at least one of the following: the size / shape of the current block, the color format, or the number of intra-frame prediction modes selected from the histogram during intra-frame prediction of the luminance block. Thereafter, the final prediction block of the chrominance block can be obtained by performing an average operation or a weighted sum operation on the N prediction blocks.

[0215] In this case, for the weighted sum operation, the weight applied to each prediction block may be determined based on the magnitude of the intra-frame prediction mode. In other words, the largest weight may be assigned to the prediction block derived based on the intra-frame prediction mode with the largest magnitude among the multiple intra-frame prediction modes, while the smallest weight may be assigned to the prediction block derived based on the intra-frame prediction mode with the smallest magnitude.

[0216] In this case, the weight assigned to each prediction block may be determined based on the ratio between the amplitudes. Alternatively, after pre-storing the weight value of each amplitude level, the weight mapped to the amplitude level of the corresponding intra prediction mode may be applied to the prediction block.

[0217] As another example, at least one intra prediction mode with a large magnitude that does not overlap with the intra prediction mode of the pre-derived chroma block may be selected from the histogram. Here, the pre-derived intra prediction mode of the chroma block may be one of the intra prediction mode candidates determined by the index information (see Table 2).

[0218] In this case, intra prediction may be performed based on the pre-derived intra prediction mode of the chrominance block and each of the at least one intra prediction mode selected from the histogram to obtain multiple prediction blocks of the chrominance block. Afterwards, a final prediction block of the chrominance block may be obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

[0219] As another example, intra prediction may be performed based on at least one default mode and each of at least one intra prediction mode derived from the histogram to obtain multiple prediction blocks of the chrominance block. Here, the default mode may include at least one of a planar mode, a DC mode, or a predefined directional mode. Thereafter, the final prediction block of the chrominance block may be obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

[0220] It is also possible to determine the intra prediction mode in units of subblocks within a chroma block.

[0221] Fig.18 is a diagram for describing an example in which an intra prediction mode is determined in units of subblocks.

[0222] For ease of description, assume the color format is 4:2:0.

[0223] As shown in the example, the chroma block may be divided into sub-blocks of a predetermined size. In the example shown, it is shown that the chroma block is divided into sub-blocks of a size of 2×2.

[0224] In addition, the reference area can be set with the sub-block of the luminance block corresponding to the sub-block of the chrominance block as the center. Fig.18 In the example shown in , it is shown that the reference area of ​​each 2×2 size sub-block in the chrominance block is specified as a 4×4 size area in the luminance block corresponding to the 2×2 size sub-block. Unlike the example shown, the reference area of ​​each sub-block can be specified in a size larger than 4×4.

[0225] Afterwards, a histogram may be derived for each sub-block. Based on the histogram of each sub-block, at least one intra-frame prediction mode may be obtained for each sub-block. Afterwards, at least one sub-prediction block may be obtained for each sub-block based on the at least one intra-frame prediction mode. When multiple sub-prediction blocks are obtained, a final prediction block for each sub-block may be obtained based on an average operation or a weighted sum operation on the sub-prediction blocks.

[0226] The grammatical terms used in the above embodiments are named only for the convenience of description.

[0227] When an embodiment described based on a decoding process or an encoding process is applied to an encoding process or a decoding process, it is included in the scope of the present disclosure. When an embodiment described in a predetermined order is changed in an order different from the description, it is also included in the scope of the present disclosure.

[0228] The above-mentioned embodiments are described based on a series of steps or flow charts, but they do not limit the time series order of the present disclosure, and if necessary, they can be performed simultaneously or in different orders. In addition, each component (as an example, unit, module, etc.) of the block diagram in the above-mentioned embodiments can be implemented as a hardware device or software, and multiple components can be combined and implemented as a hardware device or software. The above-mentioned embodiments can be recorded in a computer-readable recording medium implemented in the form of a program instruction that can be executed by various computer components. The computer-readable recording medium can include program instructions, data files, data structures, etc. individually or in combination. Hardware devices that are particularly configured to store and execute magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as optical floppy disks, and program instructions such as ROMs, RAMs, flash memories, etc. are included in computer-readable recording media. Hardware devices can be configured to operate as one or more software modules to perform processing according to the present disclosure, and software devices can be configured to operate as one or more hardware modules to perform processing according to the present disclosure.

[0229] Industrial Applicability

[0230] The embodiments described in the present disclosure may be applied to electronic devices that can encode / decode images.

Claims

1. A method for decoding an image, the method comprising: determining at least one intra prediction mode for a current block; as well as Based on the at least one intra prediction mode, performing intra prediction on the current block, The at least one intra prediction mode is determined by a histogram derived from a reference area of ​​the current block.

2. The method according to claim 1, wherein: The reference area includes a top reference area at a top position of the current block and a left reference area at a left position of the current block, and The histogram is generated based on the intra prediction mode and magnitude of each reference sample belonging to the reference region.

3. The method according to claim 2, wherein: The intra prediction mode of the reference sample is obtained based on a ratio between an absolute value of a horizontal gradient and an absolute value of a vertical gradient of the reference sample.

4. The method according to claim 3, wherein: The amplitude of the reference sample is derived as the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the reference sample.

5. The method according to claim 3, wherein: obtaining the horizontal gradient and the vertical gradient of the reference sample by applying a filter to the reference sample, and The filter is a Sobel mask or a Prewitt mask.

6. The method according to claim 2, wherein: The histogram is obtained by accumulating the amplitudes of the reference samples of each intra prediction mode.

7. The method according to claim 6, wherein: The at least one intra prediction mode is determined in descending order of amplitude values ​​accumulated in the histogram.

8. The method according to claim 7, wherein: When a plurality of intra prediction modes are selected from the histogram, intra prediction is performed based on each of the plurality of intra prediction modes to obtain a plurality of prediction blocks, and The final prediction block of the current block is obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

9. The method according to claim 1, wherein: performing intra prediction based on each of the at least one intra prediction mode and the default mode to obtain a plurality of prediction blocks, and A final prediction block of the current block is obtained by performing an average operation or a weighted sum operation on the multiple prediction blocks.

10. The method according to claim 9, wherein: The default mode includes at least one of a planar mode, a DC mode, a vertical mode, or a horizontal mode.

11. A method for encoding an image, the method comprising: determining at least one intra prediction mode for a current block; as well as Based on the at least one intra prediction mode, performing intra prediction on the current block, The at least one intra prediction mode is determined by a histogram derived from a reference area of ​​the current block.

12. The method according to claim 11, wherein: The reference area includes a top reference area at a top position of the current block and a left reference area at a left position of the current block, and The histogram is generated based on the intra prediction mode and magnitude of each reference sample belonging to the reference region.

13. The method according to claim 12, wherein: The intra prediction mode of the reference sample is obtained based on a ratio between an absolute value of a horizontal gradient and an absolute value of a vertical gradient of the reference sample.

14. The method according to claim 13, wherein: The amplitude of the reference sample is derived as the sum of the absolute value of the horizontal gradient and the absolute value of the vertical gradient of the reference sample.

15. A computer-readable recording medium storing a bit stream generated by an image encoding method, the image encoding method comprising: determining at least one intra prediction mode for a current block; as well as Based on the at least one intra prediction mode, performing intra prediction on the current block, The at least one intra prediction mode is determined by a histogram derived from a reference area of ​​the current block.

Citation Information

Cited By

  • Decoding method and encoding method for VIPM determination

    WO2026199618A1