Image encoding / decoding method and apparatus

By using reconstructed luminance blocks and luminance components to predict chrominance blocks during image encoding/decoding, the problem of high-resolution image data transmission and storage costs is solved, and the encoding/decoding efficiency and prediction accuracy are improved.

CN118901237APending Publication Date: 2024-11-05KT CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380029197.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-21
Filing Date
2023-03-20
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

When processing high resolution and high quality images, it is difficult for the prior art to effectively compress and transmit or store image data, resulting in increased transmission and storage costs.

Method used

The initial prediction parameters are adjusted to obtain a more accurate chroma block prediction block by using reconstructed luminance blocks during image encoding/decoding and prediction based on the linearity of the luminance component and chroma component.

Benefits of technology

Improve encoding/decoding efficiency and intra prediction accuracy, reducing the cost of image data transmission and storage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118901237A_ABST
    Figure CN118901237A_ABST
Patent Text Reader

Abstract

An image encoding / decoding method according to the present invention comprises the steps of: deriving an initial prediction parameter for a chroma block; adjusting the initial prediction parameters; and obtaining a prediction block of the chroma block based on the adjusted prediction parameter and the reconstructed luma block.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to methods and apparatus for processing image signals. Background Art

[0002] Recently, the demand for high-resolution and high-quality images such as HD (high-definition) images and UHD (ultra-high-definition) images has increased in various application fields. Since the amount of data increases relatively compared with existing image data as image data becomes high-resolution and high-quality, when the image data is transmitted by using a medium such as an existing wired and wireless broadband circuit or when the image data is stored by using an existing storage medium, the transmission cost and the storage cost increase. These problems caused by the image data becoming high-resolution and high-quality can be solved by using an efficient image compression technology.

[0003] There are various technologies, such as inter-frame prediction technology that uses image compression technology to predict pixel values ​​included in a current picture based on previous or subsequent pictures of the current picture, intra-frame prediction technology that predicts pixel values ​​included in a current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequencies and assigns long symbols to values ​​with low occurrence frequencies, etc., and image data can be effectively compressed and transmitted or stored by using these image compression technologies.

[0004] On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service also increases. Video compression technology for efficiently providing high-resolution and ultra-high-resolution stereoscopic image content has been discussed. Summary of the invention

[0005] Technical issues

[0006] An object of the present disclosure is to provide a method and apparatus for predicting a chrominance block by using a reconstructed luminance block when encoding / decoding an image signal.

[0007] An object of the present disclosure is to provide a method and apparatus for predicting a chrominance block based on the linearity of a luminance component and a chrominance component when encoding / decoding an image signal.

[0008] The technical effects of the present disclosure may not be limited to the above-mentioned technical effects, and ordinary technicians in the technical field to which the present disclosure belongs can clearly understand other unmentioned technical effects according to the following description.

[0009] Technical Solutions

[0010] The image decoding method according to the present disclosure includes: deriving initial prediction parameters for a chrominance block; adjusting the initial prediction parameters; and obtaining a prediction block of the chrominance block based on the adjusted prediction parameters and a reconstructed luminance block.

[0011] The image encoding method according to the present disclosure includes: deriving initial prediction parameters for a chrominance block; adjusting the initial prediction parameters; and obtaining a prediction block of the chrominance block based on the adjusted prediction parameters and a reconstructed luminance block.

[0012] In the image decoding / encoding method according to the present disclosure, the adjusted prediction parameters may be derived through parameter adjustment offsets.

[0013] In the image decoding / encoding method according to the present disclosure, the initial prediction parameters include initial weights, and the adjusted weights may be obtained by adding parameter adjustment offsets to the initial weights.

[0014] In the image decoding / encoding method according to the contents of the present disclosure, the initial prediction parameters also include an initial offset, and the adjusted weight can be obtained by adding a difference value to the initial weight, where the difference value is obtained by multiplying the average, minimum, maximum or median value of the brightness reference sample by the parameter adjustment offset.

[0015] In the image decoding / encoding method according to the present disclosure, a parameter adjustment offset is determined based on an index indicating one parameter adjustment offset candidate among a plurality of parameter adjustment offset candidates, and the index may be explicitly signaled through a bitstream.

[0016] In the image decoding / encoding method according to the contents of the present disclosure, when one of multiple parameter adjustment offset candidate sets is selected for a current block, a parameter adjustment offset is selected from multiple parameter adjustment offset candidates included in the selected parameter adjustment offset candidate set, and one of the multiple parameter adjustment offset candidate sets can be selected based on at least one of the size, shape, quantization parameter or image format of the current block.

[0017] In the image decoding / encoding method according to the contents of the present disclosure, initial prediction parameters are derived based on downsampled luminance reference samples in a reference area adjacent to a luminance block and chrominance reference samples in a reference area adjacent to a chrominance block, and the position to apply the downsampling filter in the reference area of ​​the luminance block can be determined based on the cost of each position to which the downsampling filter can be applied.

[0018] An object of the present disclosure is to provide a method and apparatus for predicting a chrominance block by using a reconstructed luminance block when encoding / decoding an image signal.

[0019] An object of the present disclosure is to provide a method and apparatus for predicting a chrominance block based on the linearity of a luminance component and a chrominance component when encoding / decoding an image signal.

[0020] In the image decoding / encoding method according to the present disclosure, down-sampled luma reference samples are classified into a plurality of groups, and an initial prediction parameter and an adjusted prediction parameter may be obtained for each of the plurality of groups.

[0021] In the image decoding / encoding method according to the present disclosure, the parameter adjustment offset may be independently determined for each of a plurality of groups.

[0022] A computer-readable recording medium storing a bit stream encoded by the image encoding method according to the present disclosure may be provided.

[0023] Technical Effects

[0024] According to the present disclosure, encoding / decoding efficiency may be improved by predicting a chrominance block using a reconstructed luminance block.

[0025] According to the present disclosure, the accuracy of intra prediction can be improved by predicting a chrominance block based on the linearity of a luma component and a chrominance component.

[0026] Effects that can be obtained according to the present disclosure are not limited to the above-mentioned effects, and other unmentioned effects can be clearly understood by a person of ordinary skill in the technical field to which the present disclosure pertains from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present disclosure.

[0028] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0029] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0030] Figure 4 and Figure 5 Examples of multiple intra-prediction modes according to the present disclosure are shown.

[0031] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0032] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0033] Figure 8A directional mode based intra prediction method according to the present disclosure is shown.

[0034] Fig. 9 A method for deriving samples at fractional positions is shown.

[0035] Fig.10 and Fig.11 It is shown that the tangent value of the angle is scaled by a factor of 32 per intra prediction mode.

[0036] Fig.12 34 is a diagram showing aspects of intra prediction when the directional mode is one of mode number 34 to mode number 34.

[0037] Fig.13 is a diagram for describing an example of generating top reference samples by interpolating left reference samples.

[0038] Fig.14 An example of performing intra prediction by using reference samples arranged in a 1D array is shown.

[0039] Fig.15 is a flow chart illustrating a method for predicting a chrominance block by using a reconstructed luma block.

[0040] Figures 16 to 18 An example of downsampling a luma block is shown.

[0041] Fig.19 is a diagram for describing an example regarding a location where downsampling is applied.

[0042] Fig. 20 An example of adjusting prediction parameters derived based on luma reference samples and chroma reference samples is shown.

[0043] Fig.21 An example is shown in which prediction parameters are adjusted by group.

[0044] Fig. 22 An example of a right reference sample and a bottom reference sample where a chroma block is derived is shown.

[0045] Mode for carrying out the invention

[0046] Since the present disclosure can be variously changed and has several embodiments, specific embodiments will be described in the accompanying drawings and described in detail. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood that the present disclosure includes all changes, equivalents or substitutes within the scope of the ideas and techniques of the present disclosure. When describing each of the accompanying drawings, similar reference numerals are used for similar parts.

[0047] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by the terms. Terms are used only to distinguish one component from other components. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component, without exceeding the scope of the rights of the present disclosure. Terms and / or combinations of multiple related input items or any item of multiple related input items.

[0048] When a component is referred to as being "linked" or "connected" to another component, it should be understood that the component may be directly linked or connected to the other component, but other components may also exist in between. On the other hand, when a component is referred to as being "directly linked" or "directly connected" to another component, it should be understood that no other components exist in between.

[0049] Because the terms used in this application are only used to describe specific embodiments, they are not intended to limit the present disclosure. Singular expressions include plural expressions unless the singular expressions clearly have different meanings in the context. In this application, it should be understood that terms such as "including" or "having" refer to the presence of characteristics, numbers, steps, movements, parts, parts or combinations thereof entered in the specification, but do not exclude the possibility of pre-adding or existing one or more other characteristics, numbers, steps, movements, parts, parts or combinations thereof.

[0050] Hereinafter, with reference to the accompanying drawings, a desired embodiment of the present disclosure will be described in more detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated description of the same components is omitted.

[0051] Figure 1 is a block diagram showing an image encoding apparatus according to an embodiment of the present disclosure.

[0052] Reference Figure 1 The image encoding device (100) may include: a picture segmentation unit (110), a prediction unit (120, 125), a transform unit (130), a quantization unit (135), a rearrangement unit (160), an entropy encoding unit (165), a dequantization unit (140), an inverse transform unit (145), a filter unit (150) and a memory (155).

[0053] because Figure 1Each of the construction units shown in is shown independently to represent different characteristic functions in the image encoding device, which does not mean that each construction unit is composed of separate hardware or one software unit. That is, since each construction unit is included for the convenience of description by listing each construction unit as each construction unit, at least two construction units in each construction unit can be combined to constitute one construction unit, or one construction unit can be divided into multiple construction units to perform functions, and even the integrated implementation and separate implementation of each construction unit are included in the scope of the rights of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0054] In addition, some components may be only optional components for improving performance, rather than essential components for performing the basic functions in the present disclosure. The present disclosure may be implemented by including only the structural units necessary for realizing the essence of the present disclosure and excluding components only for improving performance, and a structure including only essential components and excluding optional components only for improving performance is also included in the scope of the rights of the present disclosure.

[0055] The picture segmentation unit (110) may segment an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the picture segmentation unit (110), a picture may be segmented into a combination of a plurality of decoding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of a decoding unit, a prediction unit, and a transform unit according to a predetermined criterion (e.g., a cost function).

[0056] For example, one picture may be partitioned into a plurality of decoding units. In order to partition the decoding units in the picture, a recursive tree structure such as a quadtree may be used, and a decoding unit partitioned into other decoding units by using one image or a maximum coding unit as a path may be partitioned with as many child nodes as the number of the partitioned decoding units. A decoding unit that is no longer partitioned according to a specific restriction becomes a leaf node. In other words, assuming that only square partitioning is possible for one decoding unit, one decoding unit may be partitioned into up to four other decoding units.

[0057] Hereinafter, in an embodiment of the present disclosure, a decoding unit may be used as a unit for encoding, or may be used as a unit for decoding.

[0058] The prediction unit may be partitioned with at least one square or rectangular shape, etc., in the same size in one decoding unit, or may be partitioned such that any one of the prediction units partitioned in one decoding unit may have a different shape and / or size from another prediction unit.

[0059] When a prediction unit for which intra prediction is performed based on a coding unit is generated and the prediction unit is not a minimum coding unit, intra prediction may be performed without partitioning the prediction unit into a plurality of prediction units N×N.

[0060] The prediction unit (120, 125) may include an inter-frame prediction unit (120) that performs inter-frame prediction and an intra-frame prediction unit (125) that performs intra-frame prediction. It may be determined whether to perform inter-frame prediction or intra-frame prediction on the decoding unit, and detailed information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and details. For example, the prediction method, prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and may be sent to the decoding device. When a specific encoding mode is used, the original block may be encoded as is without generating a prediction block through the prediction unit (120, 125) and sent to the decoding unit.

[0061] The inter-frame prediction unit (120) can predict the prediction unit based on information about at least one picture in the previous picture or the subsequent picture of the current picture, or in some cases can predict the prediction unit based on information about some coding areas in the current picture. The inter-frame prediction unit (120) can include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0062] The reference picture interpolation unit may receive reference picture information from a memory (155) and generate pixel information equal to or less than an integer pixel in the reference picture. For luma pixels, an 8-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 4 pixels. For chrominance signals, a 4-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 8 pixels.

[0063] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. As a method for calculating a motion vector, various methods such as FBMA (block matching algorithm based on full search), TSS (three-step search), NTS (new three-step search algorithm), etc. may be used. Based on the interpolated pixel, the motion vector may have a motion vector value in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc. may be used as a motion prediction method.

[0064] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When the neighboring block in the current prediction unit is a block for performing inter-frame prediction and therefore the reference pixel is a pixel for performing inter-frame prediction, the reference pixel included in the block for performing inter-frame prediction can be used by replacing it with the reference pixel information of the surrounding blocks for performing intra-frame prediction. In other words, when the reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one reference pixel among the available reference pixels.

[0065] When performing prediction, the prediction mode of intra prediction may have a directional prediction mode using reference pixel information based on the prediction direction and a non-directional mode not using directional information. The mode for predicting luminance information may be different from the mode for predicting chrominance information, and the chrominance information may be predicted using the intra prediction mode information for predicting luminance information or the predicted luminance signal information.

[0066] In the case where the size of the prediction unit is the same as the size of the transform unit when performing intra prediction, intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit. However, if the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0067] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of AIS filter applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode in the current prediction unit may be predicted based on the intra prediction modes in the prediction units surrounding the current prediction unit. When the prediction mode in the current prediction unit is predicted by using the mode information predicted based on the surrounding prediction units, if the intra prediction mode in the current prediction unit is the same as the intra prediction mode in the surrounding prediction units, information about the prediction mode in the current prediction unit being the same as the prediction mode in the surrounding prediction units may be transmitted by using predetermined flag information, and if the prediction mode in the current prediction unit is different from the prediction mode in the surrounding prediction units, the prediction mode information of the current block may be encoded by performing entropy coding.

[0068] In addition, a residual block including information on a residual value that is a difference value between a prediction unit predicted based on the prediction unit generated in the prediction unit (120, 125) and an original block in the prediction unit may be generated. The generated residual block may be input to the transform unit (130).

[0069] The transform unit (130) can transform the original block and the residual block described below by using a transform method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT, and the residual block includes residual value information in the prediction unit generated by the prediction unit (120, 125). Whether to apply DCT, DST or KLT to transform the residual block can be determined based on intra-frame prediction mode information in the prediction unit used to generate the residual block.

[0070] The quantization unit (135) may quantize the value transformed into the frequency domain in the transformation unit (130). The quantization coefficient may be changed according to the importance or block of the image. The value calculated in the quantization unit (135) may be provided to the dequantization unit (140) and the rearrangement unit (160).

[0071] The rearrangement unit 160 may perform rearrangement on coefficient values ​​of the quantized residual value.

[0072] The rearrangement unit (160) can change the coefficients in the shape of a two-dimensional block into the shape of a one-dimensional vector by a coefficient scanning method. For example, the rearrangement unit (160) can scan from the DC coefficient to the coefficients in the high frequency domain by using a zigzag scanning method and change them into the shape of a one-dimensional vector. Depending on the size of the transform unit and the intra-frame prediction mode, instead of the zigzag scanning, vertical scanning of scanning the coefficients in the shape of a two-dimensional block along the column direction or horizontal scanning of scanning the coefficients in the shape of a two-dimensional block along the row direction can be used. In other words, which scanning method of zigzag scanning, vertical scanning, and horizontal scanning to be used can be determined according to the size of the transform unit and the intra-frame prediction mode.

[0073] The entropy coding unit (165) may perform entropy coding based on the value calculated by the rearrangement unit (160). For example, the entropy coding may use various coding methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding).

[0074] The entropy coding unit (165) can encode various information from the rearrangement unit (160) and the prediction unit (120, 125), such as residual value coefficient information and block type information in the decoding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0075] The entropy encoding unit (165) may perform entropy encoding on the coefficient values ​​in the decoding unit input from the rearrangement unit (160).

[0076] The dequantization unit (140) and the inverse transform unit (145) dequantize the value quantized in the quantization unit (135) and perform inverse transform on the value transformed in the transform unit (130). The residual value generated by the dequantization unit (140) and the inverse transform unit (145) can be combined with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0077] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0078] The deblocking filter can remove block distortion generated by the boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it can be determined whether to apply the deblocking filter to the current block based on the pixels included in several rows or columns included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter can be applied according to the required deblocking filter strength. In addition, when applying the deblocking filter, when performing horizontal filtering and vertical filtering, horizontal directional filtering and vertical directional filtering can be set to parallel processing.

[0079] The offset correction unit can correct the offset from the original image in units of pixels for the image to be deblocked. In order to perform offset correction on a specific picture, the area where the offset is to be performed can be determined after dividing the pixels included in the image into a certain number of areas, and a method of applying the offset to the corresponding area or a method of applying the offset by considering edge information of each pixel can be used.

[0080] Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing a filtered reconstructed image with an original image. After the pixels included in the image are divided into predetermined groups, filtering may be performed distinguishably by group by determining a filter to be applied to the corresponding group. Information on whether ALF is applied may be transmitted per coding unit (CU) for a luminance signal, and the shape and filter coefficient of the ALF filter to be applied may differ for each block. In addition, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to be applied.

[0081] The memory (155) may store a reconstructed block or picture calculated by the filter unit (150), and when inter-frame prediction is performed, the stored reconstructed block or picture may be provided to the prediction unit (120, 125).

[0082] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0083] Reference Figure 2 , the image decoding device (200) may include: an entropy decoding unit (210), a rearrangement unit (215), a dequantization unit (220), an inverse transform unit (225), a prediction unit (230, 235), a filter unit (240) and a memory (245).

[0084] When an image bitstream is input from the image encoding device, the input bitstream may be decoded according to a process reverse to that of the image encoding device.

[0085] The entropy decoding unit (210) can perform entropy decoding according to a process opposite to the process of performing entropy encoding in the entropy encoding unit of the image encoding device. For example, various methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), CABAC (Context Adaptive Binary Arithmetic Coding) can be applied in response to the method performed in the image encoding device.

[0086] The entropy decoding unit (210) can decode information related to intra-frame prediction and inter-frame prediction performed in the encoding device.

[0087] The rearrangement unit (215) may perform rearrangement based on a method of rearranging the bit stream entropy decoded in the entropy decoding unit (210) in the coding unit. The coefficients expressed in the form of a one-dimensional vector may be rearranged by reconstructing the coefficients in the form of a two-dimensional block. The rearrangement unit (215) may receive information related to coefficient scanning performed in the coding unit, and perform rearrangement by performing scanning in reverse based on a scanning order performed in the corresponding coding unit.

[0088] The dequantization unit (220) may perform dequantization based on the quantization parameter provided from the encoding device and the coefficient value of the rearranged block.

[0089] The inverse transform unit (225) can perform the transform performed in the transform unit, that is, the inverse transform for DCT, DST and KLT (that is, inverse DCT, inverse DST and inverse KLT) on the result of the quantization performed in the image encoding device. The inverse transform can be performed based on the transmission unit determined in the image encoding device. In the inverse transform unit (225) of the image decoding device, the transform technology (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of information such as a prediction method, a size of a current block, a prediction mode, etc.

[0090] The prediction unit (230, 235) may generate a prediction block based on information related to generation of the prediction block provided from the entropy decoding unit (210) and pre-decoded block or picture information provided from the memory (245).

[0091] As described above, when the size of the prediction unit is the same as the size of the transform unit when performing intra prediction in the same manner as the operation in the image encoding device, intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit, but when the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0092] The prediction unit (230, 235) may include: a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information such as prediction unit information, prediction mode information of an intra-frame prediction method, motion prediction related information of an inter-frame prediction method, etc. input from the entropy decoding unit (210), divide the prediction unit in the current decoding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit (230) may perform inter-frame prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of a current picture including the current prediction unit by using information required for inter-frame prediction in the current prediction unit provided from the image encoding device. Alternatively, inter-frame prediction may be performed based on information about some areas pre-reconstructed in the current picture including the current prediction unit.

[0093] In order to perform inter prediction, whether a motion prediction method in a prediction unit included in a corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or an intra block copy mode may be determined on a coding unit basis.

[0094] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that performs intra prediction, intra prediction can be performed based on intra prediction mode information in the prediction unit provided from the image encoding device. The intra prediction unit (235) may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As a part of performing filtering on the reference pixels of the current block, the AIS filter can be applied by determining whether to apply the filter according to the prediction mode in the current prediction unit. By using the prediction mode and AIS filter information in the prediction unit provided from the image encoding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0095] When the prediction mode in the prediction unit is a prediction unit that performs intra-frame prediction based on a pixel value interpolated from a reference pixel, the reference pixel interpolation unit may interpolate the reference pixel so as to generate the reference pixel in units of pixels equal to or less than an integer value. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0096] The reconstructed block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0097] Information about whether to apply a deblocking filter to a corresponding block or picture and information about whether to apply a strong filter or a weak filter when applying the deblocking filter can be provided from the image encoding device. Information related to the deblocking filter provided from the image encoding device can be provided in the deblocking filter of the image decoding device, and deblocking filtering of the corresponding block can be performed in the image decoding device.

[0098] The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image when encoding is performed, offset value information, and the like.

[0099] ALF may be applied to the decoding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoding device. Such ALF information may be provided by including it in a specific parameter set.

[0100] The memory (245) may store the reconstructed picture or block to be used as a reference picture or a reference block, and provide the reconstructed picture to the output unit.

[0101] As described above, hereinafter, in the embodiments of the present disclosure, for convenience of description, a decoding unit is used as a term of a decoding unit, but may be a unit that performs decoding as well as encoding.

[0102] In addition, since the current block represents a block to be encoded / decoded, it may represent a decoding tree block (or decoding tree unit), a decoding block (or decoding unit), a transform block (or transform unit), or a prediction block (or prediction unit), etc., according to the encoding / decoding step. In the present specification, a "unit" may represent a basic unit for performing a specific encoding / decoding process, and a "block" may represent a pixel array of a predetermined size. Unless otherwise classified, "block" and "unit" may be used interchangeably. For example, in the embodiments described later, it may be understood that a decoding block (decoding block) and a decoding unit (decoding unit) may be used interchangeably.

[0103] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0104] Reference Figure 3 , a reference line for intra prediction of a current block may be determined ( S300 ).

[0105] The current block may use one or more reference line candidates among a plurality of reference line candidates predefined in an image encoding / decoding device as a reference line for intra prediction. Here, the plurality of predefined reference line candidates may include adjacent reference lines adjacent to the current block to be decoded and N non-adjacent reference lines that are 1 sample to N samples away from the boundary of the current block. N may be an integer of 1, 2, 3 or more. In the following, for ease of description, it is assumed that the plurality of reference line candidates available for the current block include adjacent reference line candidates and three non-adjacent reference line candidates, but is not limited thereto. In other words, of course, the plurality of reference line candidates available for the current block may include four or more non-adjacent reference line candidates.

[0106] The image encoding device may determine the best reference line candidate among multiple reference line candidates and encode an index for specifying the best reference line candidate. The image decoding device may determine the reference line of the current block based on the index signaled by the bitstream. The index may specify any one of the multiple reference line candidates. The reference line candidate specified by the index may be used as the reference line of the current block.

[0107] The number of indexes signaled to determine the reference line of the current block may be 1, 2, or more. As an example, when the number of indexes signaled above is 1, the current block may perform intra-frame prediction by using only a single reference line candidate specified by the signaled index among multiple reference line candidates. Alternatively, when the number of indexes signaled above is at least 2, the current block may perform intra-frame prediction by using multiple reference line candidates specified by multiple indexes among multiple reference line candidates.

[0108] Reference Figure 3 , the intra prediction mode of the current block can be determined (S310).

[0109] The intra prediction mode of the current block may be determined among a plurality of intra prediction modes predefined in the image encoding / decoding device. Figure 4 and Figure 5 To describe multiple predefined intra prediction modes.

[0110] Figure 4 Examples of multiple intra-prediction modes according to the present disclosure are shown.

[0111] Reference Figure 4 , the plurality of intra prediction modes predefined in the image encoding / decoding device may include a non-directional mode and a directional mode. The non-directional mode may include at least one of a planar mode or a DC mode. The directional mode may include directional modes 2 to 66.

[0112] Directional mode can be compared Figure 4 The directional pattern shown is much more extended. Figure 5 An example in which the directional mode is extended is shown.

[0113] exist Figure 5 , adding mode -1 to mode -14 and mode 67 to mode 80 are shown. These directional modes may be referred to as wide-angle intra prediction modes. Whether to use the wide-angle intra prediction mode may be determined based on the shape of the current block. As an example, if the current block is a non-square block having a width greater than a height, some directional modes (e.g., numbers 2 to 15) may be transformed into wide-angle intra prediction modes between numbers 67 and 80. On the other hand, if the current block is a non-square block having a height greater than a width, some directional modes (e.g., numbers 53 to 66) may be transformed into wide-angle intra prediction modes between numbers 1 and -14.

[0114] The range of available wide-angle intra prediction modes may be adaptively determined according to the ratio of the width to the height of the current block. Table 1 shows the range of available wide-angle intra prediction modes according to the ratio of the width to the height of the current block.

[0115] [Table 1]

[0116] Width / Height Available wide-angle intra prediction mode range W / H=16 67~80 W / H=8 67~78 W / H=4 67~76 W / H=2 67~74 W / H=1 none W / H=1 / 2 -1~-8 W / H=1 / 4 -1~-10 W / H=1 / 8 -1~-12 W / H=1 / 16 -1~-14

[0117] K candidate modes (most probable modes, MPM) may be selected from among a plurality of intra prediction modes. A candidate list including the selected candidate modes may be generated. An index indicating any one of the candidate modes belonging to the candidate list may be signaled. The intra prediction mode of the current block may be determined based on the candidate mode indicated by the index. As an example, the candidate mode indicated by the index may be configured as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be determined based on the value of the candidate mode indicated by the index and a predetermined difference. The difference may be defined as the difference between the value of the intra prediction mode of the current block and the value of the candidate mode indicated by the index. The difference may be signaled by a bitstream. Alternatively, the difference may be a value predefined in an image encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag showing whether there is a mode identical to the intra prediction mode of the current block in the candidate list. As an example, when the flag is a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of the multiple candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be configured as the intra prediction mode of the current block. On the other hand, when the flag is a second value, any one of the remaining intra prediction modes may be configured as the intra prediction mode of the current block. The remaining intra prediction modes may refer to a mode excluding the candidate mode belonging to the candidate list in a plurality of predefined intra prediction modes. When the flag is a second value, an index indicating any one of the remaining intra prediction modes may be signaled. The intra prediction mode indicated by the signaled index may be configured as the intra prediction mode of the current block. The intra prediction mode of a chroma block may be selected from the intra prediction mode candidates of a plurality of chroma blocks. To this end, index information indicating one of the intra prediction mode candidates of the intra prediction mode of the chroma block may be explicitly encoded and signaled by a bitstream. Table 2 shows the intra prediction mode candidates of the chroma block.

[0118] [Table 2]

[0119]

[0120] In the example of Table 2, the direct mode (DM) refers to setting the intra prediction mode of the luminance block present at the same position as the chrominance block as the intra prediction mode of the chrominance block. As an example, if the intra prediction mode of the luminance block (luminance mode) is number 0 (planar mode) and the index indicates number 2, the intra prediction mode of the chrominance block may be determined as the horizontal mode (number 18). As an example, if the intra prediction mode of the luminance block (luminance mode) is number 1 (DC mode) and the index indicates number 0, the intra prediction mode of the chrominance block may be determined as the planar mode (number 0).

[0121] Therefore, the intra prediction mode of the chroma block can also be set to Figure 4 or Figure 5 One of the intra prediction modes shown. The intra prediction mode of the current block may be used to determine the reference line of the current block, and in this case, S310 may be performed before S300.

[0122] Reference Figure 3 , intra prediction may be performed on the current block based on the intra prediction mode and the reference line of the current block (S320).

[0123] In the following, by reference Figures 6 to 8 The intra prediction method of each intra prediction mode is described in detail. However, for ease of description, it is assumed that a single reference line is used for intra prediction of the current block, but even in the case of using multiple reference lines, the intra prediction method described later can be applied in the same / similar manner.

[0124] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0125] Reference Figure 6 , T represents a reference sample located at the upper right corner of the current block, and L represents a reference sample located at the lower left corner of the current block. P1 can be generated by interpolation in the horizontal direction. As an example, P1 can be generated by interpolating T and a reference sample located on the same horizontal line as P1. P2 can be generated by interpolation in the vertical direction. As an example, P2 can be generated by interpolating L and a reference sample located on the same vertical line as P2. As in the following equation 3, the current sample in the current block can be predicted by the weighted sum of P1 and P2.

[0126] [Formula 1]

[0127] (α×P1+β×P2) / (α+β)

[0128] In Formula 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, weights α and β may be configured to be the same, and the predicted sample of the current sample may be configured as the average of P1 and P2. When the width and height of the current block are different, weights α and β may have different values. As an example, when the width is greater than the height, a smaller value may be configured as a weight corresponding to the width of the current block, and a larger value may be configured as a weight corresponding to the height of the current block. Conversely, when the width is greater than the height, a larger value may be configured as a weight corresponding to the width of the current block, and a smaller value may be configured as a weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block may refer to β, and the weight corresponding to the height of the current block may refer to α.

[0129] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0130] Reference Figure 7 , the average value of the surrounding samples adjacent to the current block can be calculated, and the calculated average value can be configured as the prediction value of all samples in the current block. Here, the surrounding samples may include the top reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value may be calculated by using only the top reference sample or the left reference sample. As an example, when the width of the current block is greater than the height, the average value may be calculated by using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated by using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value may be calculated by using only the top reference sample of the current block. On the other hand, when the width of the current block is less than the height, the average value may be calculated by using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value may be calculated by using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated by using only left reference samples of the current block.

[0131] Figure 8 A directional mode based intra prediction method according to the present disclosure is shown.

[0132] When the intra-frame prediction mode of the current block is a directional mode, projection can be performed on the reference line according to the angle of the corresponding directional mode. When there is a reference sample at the projection position, the corresponding reference sample can be configured as a prediction sample of the current sample. If there is no reference sample at the projection position, a sample corresponding to the projection position can be generated by using one or more surrounding samples adjacent to the projection position. As an example, interpolation can be performed based on two or more surrounding samples adjacent in two directions relative to the projection position to generate a sample corresponding to the projection position. Alternatively, a surrounding sample adjacent to the projection position can be configured as a sample corresponding to the projection position. In this case, among the multiple surrounding samples adjacent to the projection position, the surrounding sample closest to the projection position can be used. The sample corresponding to the projection position can be configured as a prediction sample of the current sample.

[0133] Reference Figure 8 , for the current sample B, when projection is performed on the reference line at the corresponding position according to the angle of the intra-frame prediction mode, there is a reference sample at the projection position (i.e., the reference sample R3 at the integer position). In this case, the reference sample at the projection position can be configured as a prediction sample for the current sample B. For the current sample A, when projection is performed on the reference line at the corresponding position according to the angle of the intra-frame prediction mode, there is no reference sample at the projection position (i.e., the reference sample at the integer position). In this case, interpolation can be performed based on surrounding samples (e.g., R2 and R3) adjacent to the projection position to generate a sample (r) at a fractional position. The generated sample (r) at the fractional position can be configured as a prediction sample for the current sample A.

[0134] Fig. 9 A method for deriving samples at fractional positions is shown.

[0135] exist Fig. 9 In the example of , the variable h refers to the vertical distance (i.e., vertical distance) from the position of the prediction sample A to the reference sample line, and the variable w refers to the horizontal distance (i.e., horizontal distance) from the position of the prediction sample A to the fractional position sample. In addition, the variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and the variable x refers to the fractional position.

[0136] The variable w can be obtained as shown in Equation 2 below.

[0137] [Formula 2]

[0138] w=htanθ

[0139] Then, if we remove the integer positions from the variable w, we can finally get the fractional positions.

[0140] The fractional position samples may be generated by interpolating adjacent integer position reference samples. As an example, the fractional position reference samples at the x position may be generated by interpolating integer position reference samples R2 and integer position reference samples R3.

[0141] When deriving fractional position samples, a scaling factor can be used to avoid real number operations. As an example, when the scaling factor f is set to 32, such as Figure 8 In the example shown in (b), the distance between adjacent integer reference samples can be set to 32 instead of 1.

[0142] Furthermore, the tangent value of the angle θ determined according to the directionality of the intra prediction mode may also be scaled by using the same scaling factor (eg, 32).

[0143] Fig.10 and Fig.11 It is shown that the tangent value of the angle is scaled by a factor of 32 per intra prediction mode.

[0144] Fig.10 shows the scaling results of the tangent value for non-wide angle intra prediction mode, and Fig.11 The scaling results of the tangent values ​​for the wide angle intra prediction mode are shown.

[0145] If the tangent value (tanθ) of the angle value used for the intra prediction mode is positive, intra prediction can be performed by using only one of the reference samples belonging to the top line of the current block (i.e., top reference samples) or the reference samples belonging to the left line of the current block (i.e., left reference samples). Meanwhile, when the tangent value of the angle value used for the intra prediction mode is negative, both the reference samples at the top position and the reference samples at the left position are used.

[0146] In this case, to simplify the implementation, the left reference samples may be projected to the top, or may be projected to the left to arrange the reference samples in a 1D array form, and intra prediction may be performed using the reference samples in the 1D array form.

[0147] Fig.12 34 is a diagram showing aspects of intra prediction when the directional mode is one of mode number 34 to mode number 34.

[0148] When the intra prediction mode of the current block is one of mode number 34 to mode number 49, intra prediction is performed by using not only the top reference sample of the current block but also the left reference sample. Fig.12 In the example shown, the reference samples of the top line may be generated by copying the reference samples at the left position of the current block to the position of the top line or by interpolating the reference samples at the left position.

[0149] As an example, when a reference sample of position A on the top of the current block is obtained, projection may be performed at position A on the top line to the left line of the current block in consideration of the directionality of the intra-prediction mode of the current block. If the projection position is referred to as a, the value corresponding to position a may be copied, or a fractional position value corresponding to a may be generated and set to the value of position A. As an example, if position a is an integer position, the value of position A may be generated by copying the integer position reference sample. On the other hand, when position a is a fractional position, the reference sample at the top position of position a and the reference sample at the bottom position of position a may be interpolated, and the interpolated value may be set to the value of position A. Meanwhile, at position A on the top of the current block, the direction of the projection to the left line of the current block may be parallel and opposite to the direction of the intra-prediction mode of the current block.

[0150] Fig.13 is a diagram for describing an example of generating top reference samples by interpolating left reference samples.

[0151] exist Fig.13 , variable h represents the horizontal distance between position A on the top line and position a on the left line. Variable w represents the vertical distance between position A on the top line and position a on the left line. In addition, variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and variable x refers to a fractional position.

[0152] The variable h can be obtained as shown in Equation 3 below.

[0153] [Formula 3]

[0154] h=w / tanθ

[0155] Afterwards, if the integer positions are removed from the variable h, the fractional positions can be finally obtained.

[0156] When deriving fractional position samples, scaling factors can be used to avoid real number operations. As an example, the tangent of the variable θ can be scaled by using a scaling factor f1. Here, since the direction of the projection to the left line is parallel and opposite to the directional prediction mode, we can use Fig.10 and Fig.11 Scaled tangent values ​​shown.

[0157] When the scaling factor f1 is applied, Equation 3 may be modified and used as in Equation 4 below.

[0158] [Formula 4]

[0159] h=(w*f1*f2) / (f1*tanθ)

[0160] In the same manner as above, a 1D reference sample array may be configured with only reference samples belonging to the top line. Thus, intra prediction of the current block may be performed by using only the top reference samples comprising the 1D array.

[0161] Fig.14 An example of performing intra prediction by using reference samples arranged in a 1D array is shown.

[0162] like Fig.14 In the example shown, the left reference samples may be projected to generate the top reference samples, and thus, the prediction samples of the current block may be obtained by using only the reference samples belonging to the top line.

[0163] and Fig.12 and Fig.14 In contrast to what is shown in , the top reference sample can be projected to the left line to configure the 1D reference sample array with only the reference samples belonging to the left line. Specifically, for pattern numbers 19 to 33 of the directional patterns in which the tangent value (tan θ) of the angle of the directional pattern is negative, the reference samples belonging to the top line can be projected to the left line to generate the left reference samples.

[0164] Prediction of the chrominance block may be performed by using the reconstructed luminance block. As described above, prediction using different color components may be referred to as inter-component prediction.

[0165] Fig.15 is a flow chart illustrating a method for predicting a chrominance block by using a reconstructed luma block.

[0166] Reference Fig.15 First, for the prediction of the chroma block, a prediction parameter may be derived (S1510). In this case, the prediction parameter may be derived in different ways according to the image format of the picture. The image format shows the chroma subsampling rate, and it may be determined as 4:4:4, 4:2:2, or 4:2:0.

[0167] If the image format is not 4:4:4, the luma blocks can be downsampled and resized to match the size of the chroma blocks.

[0168] Figures 16 to 18 An example of downsampling a luma block is shown.

[0169] For ease of description, assume that the image format is 4:2:0.

[0170] When the image format is 4:2:0, such as Fig.16In the example shown, the size of the chrominance block corresponding to the 4×4 luminance block is 2×2. In this case, the 4×4 luminance block can be reduced to 2×2 by applying a downsampling filter to the luminance block. The following equation 5 shows the application of the downsampling filter.

[0171] [Formula 5]

[0172] Downsampled_Luma[w][h]=(Luma[2*w][2*h-1]+Luma[2*w-1][2*h]+4*Luma[2*w][2*h]+Luma[2*w+1][2*h]+Luma[2*w][2*h+1]+4)》3

[0173] In Formula 5, Downsampled_Luma refers to the sample value in the downsampled luma block, and Luma refers to the value of the luma sample before downsampling. For example, Luma[0][0] may represent the position of the upper left sample in the luma block before downsampling. Since the size of the downsampled luma block is 2×2, the variables w and h representing the coordinates of the sample may have values ​​of 0 to 1, respectively.

[0174] When the down-sampling filter is applied according to Formula 5, the value of the down-sampled brightness sample can be obtained by applying a cross-shaped down-sampling filter to the brightness sample. For example, the value of the down-sampled brightness sample at the position of (0, 0) can be obtained by applying the down-sampling filter to the brightness sample at the position of (0, 0), the top brightness sample at the position of (0, 0), the left brightness sample at the position of (0, 0), the bottom brightness sample at the position of (0, 0), and the right brightness sample at the position of (0, 0).

[0175] Can be applied to Fig.17 The shapes shown are different for downsampling filters. For example, downsampled luma samples may be obtained by applying a 1D filter, a rectangular filter, or a square filter. The 1D filter may be sized 1×3 or 3×1, the rectangular filter may be sized 2×3 or 3×2, and the square filter may be sized 2×2 or 3×2.

[0176] The filter shapes can be predefined in the encoder and decoder.

[0177] Alternatively, the shape of the filter may be adaptively determined based on at least one of the size / shape of the current block, the intra prediction mode applied to the luma block, whether the positions of the chroma samples match those of the luma samples, or the image format.

[0178] Alternatively, information indicating one of a plurality of filter candidates may be encoded and signaled.

[0179] Alternatively, the filter type may be different depending on the downsampling position. For example, a 1D filter or a rectangular filter may be applied to luma samples located at the boundary of a luma block, while a cross filter may be applied to luma samples not located at the boundary of a luma block.

[0180] like Fig.17 As shown, the downsampling filter can be applied to positions where both the x-axis coordinate and the y-axis coordinate are even numbers.

[0181] Can be used with Fig.17 The application positions shown in set the application positions of the downsampling filters differently. Fig.18 Various examples of where the downsampling filter may be applied are shown.

[0182] After predefining a plurality of candidates related to the downsampling application position, one of the plurality of candidates may be selected. Fig.18 (a) to Fig.18 After the examples in (d) are defined as a plurality of candidates, index information indicating one of the plurality of examples may be encoded and signaled.

[0183] Alternatively, one of the multiple candidates may be selected based on whether the position of the chroma samples matches the position of the luma samples.

[0184] The down-sampling filter may also be applied to reference samples around the luma block. Specifically, the down-sampled luma reference samples may be obtained by applying the down-sampling filter to at least one of a top reference region adjacent to the top of the luma block or a left reference region adjacent to the left.

[0185] The same number of down-sampled luma reference samples as the number of reference samples included in the reference region of the chroma block may be obtained.

[0186] Meanwhile, the inter-component prediction mode may be divided into a top inter-component prediction mode, a left inter-component prediction mode, and a top and left inter-component prediction mode according to the configuration of the reference region. When the top inter-component prediction mode is selected, the reference region of the luminance block and the chrominance block includes only the top reference region. When the left inter-component prediction mode is selected, the reference region of the luminance block and the chrominance block includes only the left reference region. When the top and left inter-component prediction mode is selected, the reference region of the luminance block and the chrominance block may include the top reference region and the left reference region.

[0187] Information indicating which of the top inter-component prediction mode, the left inter-component prediction mode, and the top and left inter-component prediction mode is applied to the current block may be explicitly encoded and signaled. As an example, index information indicating the type of the inter-component prediction mode may be encoded and signaled.

[0188] Alternatively, one of the top component inter prediction mode, the left component inter prediction mode, and the top and left component inter prediction mode may be selected based on the size / shape of the current block, whether the current block borders the boundary of a CTU or a picture, or at least one of the intra prediction modes applied to the luma block.

[0189] For convenience of description, in an embodiment described later, it is assumed that the reference regions of the luminance block and the chrominance block include a top reference region and a left reference region.

[0190] The shape of the down-sampling filter applied to the reference area of ​​the luma block may be the same as the down-sampling filter applied to the luma block. Alternatively, the shape of the down-sampling filter applied to the reference area of ​​the luma block may be different from the down-sampling filter applied to the luma block. Alternatively, the shape of the down-sampling filter applied to the top reference area of ​​the luma block may be different from the shape of the down-sampling filter applied to the left reference area of ​​the luma block.

[0191] Meanwhile, the positions where downsampling is applied within the reference region may be predefined in the encoder and the decoder.

[0192] As another example, the decoder may itself determine where to apply downsampling in the reference region in the same manner as the encoder.

[0193] Fig.19 is a diagram for describing an example regarding a location where downsampling is applied.

[0194] When the image format is 4:2:0, a chroma block of size 1×1 corresponds to a luma block of size 2×2. Therefore, a downsampling filter can be applied to one position of four luma reference samples to derive a downsampled luma reference sample corresponding to the chroma reference sample.

[0195] When four luma reference samples corresponding to one chroma reference sample are referred to as A to D, the cost of each position may be calculated after downsampling is performed on each of the positions A to D in the reference area. Here, the cost of a specific position may be derived based on the sum of differences between a downsampled luma reference sample obtained by applying a downsampling filter centered at the corresponding position and a chroma reference sample corresponding to the corresponding position or the sum of the absolute values ​​of the differences. Therefore, the cost derived based on the sum of the absolute values ​​of the differences may be referred to as a sum of differences (SAD).

[0196] Then, the position with the lowest cost is determined as the optimal position, and the process of deriving prediction parameters to be described later can be performed by using the downsampled brightness samples at the optimal position.

[0197] Alternatively, information indicating one of a plurality of positions where the downsampling filter may be applied may be encoded and signaled. Fig.19 In the example shown, an index indicating one of the positions A to D may be encoded and signaled. To this end, the encoder may obtain a prediction parameter for each of a plurality of positions to which a downsampling filter may be applied, and may encode and signal an index indicating a position for deriving an optimal prediction parameter among the plurality of prediction parameters. Here, the optimal prediction parameter may represent a rate-distortion optimization (RDO) or cost of each prediction parameter.

[0198] Meanwhile, determining the best downsampling application position in the top reference region may be independent of determining the best downsampling application position in the left reference region. In this case, the best downsampling application position in the top reference region may be different from the best downsampling application position in the left reference region.

[0199] The reference samples of the chroma block and the downsampled luma reference samples may be used to derive prediction parameters for the chroma block. The prediction parameters may include a weight α and an offset β. The prediction parameters may be derived by using a least squares method or the like.

[0200] As an example, the weight α and the offset β may be derived based on the linearity of the maximum and minimum values ​​of the downsampled luma reference samples with the maximum and minimum values ​​of the chroma reference samples.

[0201] In this case, the prediction parameters can be derived by using only the chroma reference samples at the predefined positions and the downsampled luma reference samples corresponding to the chroma reference samples. In this case, the process of deriving the prediction parameters can be simplified, thereby reducing the complexity of the encoder and decoder. As an example, the prediction parameters can be derived by using the chroma reference samples at the positions shown in the following equation 6.

[0202] [Formula 6]

[0203] (-1, W / 4), (-1, 3W / 4), (H / 4, -1), (3H / 4, -1)

[0204] In this example, W and H represent the width and height of the chroma block, respectively. According to this example, the prediction parameters can be derived by using four chroma reference samples and four down-sampled luma reference samples corresponding thereto.

[0205] The prediction parameters may also be obtained by using reference samples at positions different from this example. As an example, the positions of the reference samples may be determined as in Equations 7 and 8 below.

[0206] [Formula 7]

[0207] (-1, W / 4), (-1, 3W / 4), (-1, 5W / 4), (-1, 7W / 4)

[0208] [Formula 8]

[0209] (H / 4,-1), (3H / 4,-1), (5H / 4,-1), (7H / 4,-1)

[0210] After predefining multiple candidates for the position of the reference sample, one of the multiple candidates may be selected. As an example, after setting each of the examples of Equations 6 to 8 above as a position candidate, the reference sample may be selected according to one of the multiple position candidates.

[0211] Information for selecting one of the plurality of candidate positions may be encoded and signaled. As an example, an index indicating one of the plurality of position candidates may be encoded and signaled.

[0212] Alternatively, one of the plurality of position candidates may be adaptively selected based on at least one of the size / shape of the current block, the color format, or whether the position of the chroma sample matches the position of the luma sample.

[0213] As an example, if the current block is square, the prediction parameter may be derived by using the position candidate in Equation 6. On the other hand, if the current block is non-square, the prediction parameter may be derived by using the position candidate in Equation 7 or Equation 8. As an example, if the current block has a non-square shape with a width greater than a height, the position candidate in Equation 7 may be used, and if the current block has a non-square shape with a height greater than a width, the position candidate in Equation 8 may be used.

[0214] Once the prediction parameters are obtained, the prediction samples of the chrominance block can be obtained based on the downsampled luma samples (S1520). As an example, the prediction samples of the chrominance block can be obtained according to the following equation 9.

[0215] [Formula 9]

[0216] PredChroma[w][h]=α×Downsampled_luma[w][h]+β,

[0217] 0≤w<W,0≤h<H

[0218] In Equation 9, PredChroma represents a prediction sample of a chroma block, and Downsampled_Luma represents a downsampled luma sample at a position corresponding to the chroma prediction sample.

[0219] Meanwhile, if the image format is 4:4:4, the above-mentioned downsampling process can be omitted. In other words, if the image format is 4:4:4, the process of downsampling the reconstructed samples in the luminance block and the process of downsampling the reference samples of the luminance block can be omitted.

[0220] As another example, regardless of the image format, the downsampling filter may not be applied to the reference area of ​​the luma block. In other words, when deriving prediction parameters, instead of using the minimum and maximum values ​​in the downsampled luma reference samples, the minimum and maximum values ​​in the luma reference samples may be used.

[0221] In an example described later, depending on whether a down-sampling filter is applied to a reference area of ​​a luma block, a luma reference sample may refer to a down-sampled luma reference sample or a luma reference sample to which a down-sampling filter is not applied.

[0222] Instead of using the prediction parameters derived by using the luma reference samples and the chroma reference samples as the final prediction parameters, the final prediction parameters may be derived by adjusting the derived prediction parameters.

[0223] Specifically, the prediction parameter obtained by using the luma reference sample and the chroma reference sample can be set as the prediction value of the prediction parameter (ie, the initial prediction parameter). Afterwards, the final prediction parameter (ie, the adjusted prediction parameter) can be obtained by adding the difference of the prediction parameter to the prediction value of the prediction parameter.

[0224] For example, a weight α obtained based on a luma reference sample and a chroma reference sample may be set as a weight prediction value, and a weight difference value may be added to the weight prediction value to obtain a final weight α′. In addition, an offset β obtained based on a luma reference sample and a chroma reference sample may be set as an offset prediction value, and an offset difference value may be added to the offset prediction value to obtain a final offset β′.

[0225] The information about the weight difference and the offset difference may be encoded and signaled separately. For example, first parameter offset information showing the weight difference and second parameter offset information showing the offset difference may be encoded and signaled separately.

[0226] Alternatively, only a single parameter adjustment offset may be explicitly encoded and signaled, and the weight difference and the offset difference may be derived based on the parameter adjustment offset. The following Equation 10 shows an example of deriving the final prediction parameters of the chroma block.

[0227] [Formula 10]

[0228] α′=α+off

[0229] β′=β-(off×Ym)

[0230] In equation 10, off represents the parameter adjustment offset. The parameter adjustment offset may be explicitly encoded and signaled. α represents the weight prediction value, and α′ represents the final weight. β represents the offset prediction value, and β′ represents the final offset. Ym represents the average value of the luma reference sample.

[0231] It is also possible to set the median, minimum, or maximum value of the luminance reference samples to the variable Ym.

[0232] Fig. 20 An example of adjusting prediction parameters derived based on luma reference samples and chroma reference samples is shown.

[0233] Fig. 20 (a) shows prediction parameters derived based on luma reference samples and chroma reference samples. Fig. 20 (b) shows the adjustment of the offset based on the parameter Fig. 20 The results of the prediction parameters are shown in (a).

[0234] The value of the parameter adjustment offset may be encoded and signaled as is.

[0235] As another example, after predefining a plurality of parameter adjustment offset value candidates, information indicating the parameter adjustment offset value applied to the current block may be encoded and signaled. The information may be an index indicating one of the plurality of candidates. As an example, in the encoder and the decoder, a plurality of candidates having a configuration such as (-4, -3, -2, -1, 0, 1, 2, 3, 4) may be predefined.

[0236] Alternatively, at least one of the number or type of parameter adjustment offset value candidates that can be selected by the current block can be determined based on at least one of the size / shape of the current block, the quantization parameter of the current block, the intra-frame prediction mode of the luma block, whether the luma block is encoded by intra-frame mode or inter-frame mode, the image format, the number of reference samples used to derive prediction parameters, the difference between the maximum (or minimum) value of the downsampled luma reference samples and the maximum (or minimum) value of the chroma reference samples, or the bit depth of the samples.

[0237] As an example, multiple parameter adjustment offset candidate sets may be predefined in the encoder and the decoder. Each of the multiple parameter adjustment offset candidate sets may include multiple parameter adjustment offset candidates. In this case, in the parameter adjustment offset candidate sets, at least one of the number or type of parameter adjustment offset candidates may be different.

[0238] As an example, the first parameter adjustment offset candidate set can be configured as (-4, -3, -2, -1, 0, 1, 2, 3, 4), the second parameter adjustment offset candidate set can be configured as (-7, -5, -3, -1, 0, 1, 3, 5, 7), and the third parameter adjustment offset candidate set can be configured as (-8, -6, -4, -2, 0, 2, 4, 6, 8).

[0239] Based on the conditions listed above, one of the multiple parameter adjustment offset candidate sets may be selected for the current block. For example, when the difference between the maximum value (or minimum value) of the downsampled luma reference sample and the maximum value (or minimum value) of the chroma reference sample is greater than a first threshold, the first parameter adjustment offset candidate set may be selected, and when the difference is not greater than the first threshold but greater than the second threshold, the second parameter adjustment offset candidate set may be selected. When the difference is also not greater than the second threshold, the third parameter adjustment offset candidate set may be selected.

[0240] Alternatively, information indicating one parameter adjustment offset candidate set among a plurality of parameter adjustment offset candidate sets may be encoded and signaled. The information may be an index indicating one parameter adjustment offset candidate set among a plurality of parameter adjustment offset candidate sets.

[0241] Thereafter, a parameter adjustment offset candidate for the current block may be selected among the parameter adjustment offset candidates included in the selected parameter adjustment offset candidate set.

[0242] The number of parameter adjustment offset candidate sets may be different from the number of parameter adjustment offset candidate sets described. In other words, in the above example, three parameter adjustment offset candidate sets are described, but there may be fewer or more parameter adjustment offset candidate sets.

[0243] In addition, the configuration of the parameter adjustment offset candidate set is not limited to the described example. For example, the parameter adjustment offset value may be a value expressed as a power of 2. In this case, the parameter adjustment offset candidate set may include a value expressed as a power of 2.

[0244] Alternatively, the parameter adjustment offset candidate set may include a value corresponding to an exponent (ie, N) of a value expressed as a power of 2 (ie, 2^N). In this case, the parameter adjustment offset value may be obtained by shifting a constant 1 to the left by a value indicated by a candidate selected from the parameter adjustment offset candidate set.

[0245] According to the sample value, the luma reference samples can be classified into a plurality of groups, and prediction parameters can be derived separately for each group. For example, if the average value of the luma reference samples is Ym, some luma reference samples having a value greater than the variable Ym among the luma reference samples can be classified into the first group, and other luma reference samples can be classified into the second group. The luma reference samples and chroma reference samples at the same position are classified in the same manner as the classification result of the luma reference samples. As an example, when the luma reference sample is classified into the first group, the chroma reference sample corresponding to the luma reference sample is also classified into the first group.

[0246] Thereafter, a first prediction parameter may be derived by using a chroma reference sample belonging to the first group and a luma reference sample belonging to the first group, and a second prediction parameter (e.g., weight α2 and offset β2) may be derived by using a chroma reference sample belonging to the second group and a luma reference sample belonging to the second group.

[0247] Depending on whether the value of the luma sample corresponding to the chroma sample to be predicted belongs to the first group or the second group, the chroma prediction sample can be obtained by using the first prediction parameter or the second prediction parameter. For example, if the luma sample corresponding to the chroma sample to be predicted belongs to the first group, the chroma prediction sample can be obtained based on the following formula 11.

[0248] [Formula 11]

[0249] PredChroma[w][h]=α1×Downsampled_Luma[w][h]+β1,

[0250] Downsampled_Luma[w][h] <Ym

[0251] If the luma sample corresponding to the chroma sample to be predicted belongs to the second group, the chroma prediction sample can be obtained based on the following Equation 12.

[0252] [Formula 12]

[0253] PredChroma[w,][h]=α2×Downsampled_Luma[w][h]+β2,

[0254] Downsampled_Luma[w][h] <Ym

[0255] Parameter adjustment offsets can be used when the forecast parameters are derived by group. In this case, the forecast parameters for each group can be adjusted by the parameter adjustment offset.

[0256] Fig.21 An example is shown in which prediction parameters are adjusted by group.

[0257] The first prediction parameter may be adjusted based on the average value Ym1 of the luma reference samples belonging to the first group, and the second prediction parameter may be adjusted based on the average value Ym2 of the luma reference samples belonging to the second group.

[0258] Information about parameter adjustment offsets may be encoded and signaled in groups.

[0259] Alternatively, information about parameter adjustment offsets may be signaled at a block level (e.g., a decoding block). In this case, each group may share information about parameter adjustment offsets signaled at a block level. In other words, the parameter adjustment offsets may be the same between groups.

[0260] Alternatively, information about parameter adjustment offsets may be encoded and signaled at the block level, while information about the difference in parameter adjustment offsets may be encoded and signaled per group. In this case, the parameter adjustment offset for a particular group may be derived by adding the parameter adjustment offset difference signaled for the corresponding group to the parameter adjustment offset signaled at the block level.

[0261] Alternatively, the parameter adjustment offset may be encoded and signaled for a first group of the plurality of groups, and the difference from the parameter adjustment offset of the first group may be encoded and signaled for a second group.

[0262] In the above embodiment, it is described that the adjustment of the prediction parameter includes the adjustment of the weight and the adjustment of the offset. Different from the description, only the weight or only the offset of the prediction parameter can be adjusted. As an example, in the example of formula 10, the weight can be adjusted by adjusting the offset only by using the parameter, or the offset can be adjusted by adjusting the offset only by using the parameter.

[0263] In passing Fig.19 In the described embodiments, it is explained that one of the positions where the down-sampling filter may be applied may be selected based on cost or one of the positions where the down-sampling filter may be applied may be selected based on signaled information.

[0264] Unlike the description, the prediction parameters can be obtained by applying a down-sampling filter to each position to which the down-sampling filter can be applied. For example, the first prediction parameter can be obtained based on the down-sampled brightness reference sample obtained by applying the down-sampling filter to position A, and the second prediction parameter can be obtained by applying the down-sampled brightness reference sample obtained by applying the down-sampling filter to position B.

[0265] A reference sample for intra prediction of a chroma block can be obtained based on the reconstructed luma sample, and the intra prediction of the chroma block can be performed by using the obtained reference sample. Specifically, based on the reconstructed luma sample, at least one of the right reference sample or the bottom reference sample of the chroma block can be obtained.

[0266] Fig. 22 An example of a right reference sample and a bottom reference sample where a chroma block is derived is shown.

[0267] According to the encoding / decoding order, there are no reconstructed samples on the right and bottom of the chroma block. On the other hand, since the luma block is encoded / decoded before the chroma block, the reconstructed samples may exist on the right and bottom of the luma block. Therefore, at least one of the right reference sample or the bottom reference sample of the chroma block may be obtained by using the reconstructed luma sample.

[0268] As an example, the reconstructed luma samples or downsampled luma samples corresponding to the right or bottom reference samples may be set to the values ​​of the right or bottom reference samples.

[0269] Alternatively, after deriving prediction parameters (i.e., weight α and offset β) by using at least one of the top reference region or the left reference region of the luma block, the right or bottom reference sample may be derived based on the derived prediction parameters. Specifically, the right or bottom reference sample may be derived by applying weight α and offset β to a reconstructed luma sample or a downsampled luma sample corresponding to the right or bottom reference sample.

[0270] Also, when using the right and / or bottom reference samples, Figure 4 As an example, the intra prediction can be performed in more directional modes as shown in . Figure 4 One of the directional modes opposite to the directional mode shown is applied to the intra prediction of the chroma block. In other words, as in this embodiment, when generating right and bottom reference samples for the chroma block, the number of directional modes that can be used for the chroma block can be greater than the number of directional modes that can be used for the luminance block.

[0271] Meanwhile, the chroma prediction samples may be obtained by applying a convolution filter to the reconstructed luma block. Whether to obtain the chroma prediction samples by applying a convolution filter may be adaptively determined based on at least one of the size / shape of the current block, the intra prediction mode of the luma block, the image format, or the number of reference samples included in the reference area.

[0272] Alternatively, predicting the chroma block by using a convolution filter may be defined as one of the inter-component prediction modes. In this case, according to the index information, it may be determined whether predicting the chroma block by using a convolution filter is selected among a plurality of inter-component prediction modes.

[0273] The convolution filter may have a size of 7 taps or 5 taps. The convolution filter may be applied to a luma sample at the same position as the chroma sample to be predicted and at least one neighboring luma sample that is adjacent to the luma sample.

[0274] Equation 13 shows an example of obtaining chrominance prediction samples by applying a convolution filter.

[0275] [Formula 13]

[0276] PredChroma=c0C+c1N+c2S+c3E+c4W+c5P+c6B

[0277] In the above formula 13, C represents a luma sample at the same position as the chroma prediction sample, N represents a top luma sample adjacent to the top of the luma sample, S represents a bottom luma sample adjacent to the bottom of the luma sample, E represents a right luma sample adjacent to the right side of the luma sample, and W represents a left luma sample adjacent to the left side of the luma sample.

[0278] P can be obtained by squaring the luma sample at the same position as the chroma prediction sample. As an example, P can be derived as in Equation 14 below.

[0279] [Formula 14]

[0280] P=(C*C+midVal)》bitDepth

[0281] In the above formula 14, midVal represents the median value of the sample expression range in bit depth.

[0282] B represents an offset value. As an example, B may be set to the median value (ie, midVal) of the sample expression range per bit depth.

[0283] Meanwhile, if there is an unavailable luma sample among the luma samples indicated by N, S, E, and W, the chroma prediction sample may be obtained by erasing the unavailable luma sample and the luma sample in the opposite direction to the above formula 13. As an example, if the luma sample at position N is unavailable (for example, beyond the boundary of the picture), the chroma prediction sample may be obtained by erasing the term including N and S from the above formula.

[0284] The grammatical terms used in the above embodiments are only named for the convenience of description.

[0285] When an embodiment described based on a decoding process or an encoding process is applied to an encoding process or a decoding process, it is included in the scope of the present disclosure. When an embodiment described in a predetermined order is changed in an order different from the description, it is also included in the scope of the present disclosure.

[0286] The above disclosure is described based on a series of steps or flow charts, but it does not limit the time series order of the present disclosure, and if necessary, it can be performed simultaneously or in different orders. In addition, each component (e.g., unit, module, etc.) of the block diagram in the above-mentioned embodiment can be configured as a hardware device or software, and a plurality of components can be combined and implemented as a hardware device or software. The above-mentioned embodiment can be recorded in a computer-readable recording medium implemented in the form of a program instruction that can be executed by various computer components. The computer-readable recording medium can include program instructions, data files, data structures, etc. individually or in combination. Hardware devices that are particularly configured to store and execute magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as optical floppy disks, and program instructions such as ROMs, RAMs, flash memories, etc. are included in computer-readable recording media. The hardware device can be configured to operate as one or more software modules to perform processing according to the present disclosure, and the software device can be configured to operate as one or more hardware modules to perform processing according to the present disclosure.

[0287] Industrial Applicability

[0288] The embodiments of the present disclosure may be applied to electronic devices that encode / decode images.

Claims

1. A method for decoding an image, the method comprising: include: deriving initial prediction parameters for the chrominance block; adjusting the initial prediction parameters; as well as A prediction block for the chrominance block is obtained based on the adjusted prediction parameters and the reconstructed luminance block.

2. The method according to claim 1, in: The adjusted prediction parameters are derived by means of parameter adjustment offsets.

3. The method according to claim 2, in: The initial prediction parameters include initial weights, and Adjusted weights are derived by adding the parameter adjustment offsets to the initial weights.

4. The method according to claim 3, in: The initial prediction parameters also include an initial offset, and The adjusted weights are obtained by adding a difference value obtained by multiplying the average, minimum, maximum or median value of the luma reference samples by the parameter adjustment offset to the initial weights.

5. The method according to claim 2, in: The parameter adjustment offset is determined based on an index indicating one of a plurality of parameter adjustment offset candidates, and the index is explicitly signaled through a bitstream.

6. The method according to claim 5, in: When one of the plurality of parameter adjustment offset candidate sets is selected for the current block, the parameter adjustment offset is selected from the plurality of parameter adjustment offset candidates included in the selected parameter adjustment offset candidate set, and One of the plurality of parameter adjustment offset candidate sets is selected based on at least one of a size, a shape, a quantization parameter, or an image format of the current block.

7. The method according to claim 1, in: deriving the initial prediction parameters based on downsampled luma reference samples in a reference area adjacent to the luma block and chroma reference samples in a reference area adjacent to the chroma block, and A position in the reference area of ​​the luma block to apply the down-sampling filter is determined based on a cost at each position to which the down-sampling filter is applicable.

8. The method according to claim 7, in: classifying the downsampled luma reference samples into a plurality of groups, and For each group in the plurality of groups, the initial prediction parameters and the adjusted prediction parameters are obtained.

9. The method according to claim 8, in: A parameter adjustment offset is determined independently for each group in the plurality of groups.

10. A method for encoding an image, the method comprising: include: deriving initial prediction parameters for the chrominance block; adjusting the initial prediction parameters; as well as A prediction block for the chrominance block is obtained based on the adjusted prediction parameters and the reconstructed luminance block.

11. The method according to claim 10, in: The initial prediction parameters are derived based on luma reference samples adjacent to the luma block and chroma reference samples adjacent to the chroma block.

12. The method according to claim 10, in: The adjusted prediction parameters are derived by means of parameter adjustment offsets.

13. The method according to claim 12, in: The initial prediction parameters include initial weights, Adjusted weights are derived by adding the parameter adjustment offsets to the initial weights.

14. The method according to claim 13, in: The initial prediction parameters also include an initial offset, and The adjusted weights are derived by adding a difference value obtained by multiplying the average, minimum, maximum, or median value of the luma reference samples by the parameter adjustment offset to the initial weights.

15. A computer-readable recording medium storing a bit stream generated by an image encoding method, the image encoding method include: The step of deriving initial prediction parameters for chrominance blocks; The step of adjusting the initial prediction parameters; as well as The step of obtaining a prediction block for the chrominance block based on the adjusted prediction parameters and the reconstructed luminance block.