Image encoding / decoding method and recording medium storing bit stream

Abstract

An image encoding / decoding method according to the present invention may comprise the steps of: collecting prediction information from a neighboring block of a current block; deriving a prediction parameter of the current block based on the collected prediction information; and obtaining prediction information of each sub-block in the sub-blocks in the current block based on the prediction parameters.

Description

Technical Field

[0001] The present disclosure relates to methods and apparatus for processing image signals. Background Art

[0002] Recently, the demand for high-resolution and high-quality images such as HD (high-definition) images and UHD (ultra-high-definition) images has increased in various application fields. As image data becomes high-resolution and high-quality, the amount of data increases relatively compared to existing image data, so when the image data is transmitted by using a medium such as an existing wired and wireless broadband circuit or when the image data is stored by using an existing storage medium, the transmission cost and the storage cost increase. These problems generated as image data becomes high-resolution and high-quality can be solved by using efficient image compression technology.

[0003] There are various technologies, such as inter-frame prediction technology that uses image compression technology to predict pixel values ​​included in a current picture based on previous or subsequent pictures of the current picture, intra-frame prediction technology that predicts pixel values ​​included in a current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequencies and assigns long symbols to values ​​with low occurrence frequencies, etc., and image data can be effectively compressed and transmitted or stored by using these image compression technologies.

[0004] On the other hand, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service also increases. Video compression technology for efficiently providing high-resolution and ultra-high-resolution stereoscopic image content has been discussed. Summary of the invention

[0005] Technical issues

[0006] The present disclosure provides a method and apparatus for deriving prediction information in units of sub-blocks based on prediction parameters when encoding / decoding an image signal.

[0007] The present disclosure provides a method and apparatus for deriving a prediction parameter of a current block based on prediction information of neighboring blocks adjacent to the current block when encoding / decoding an image signal.

[0008] The technical effects of the present disclosure may not be limited to the above-mentioned technical effects, and ordinary technicians in the technical field to which the present disclosure belongs can clearly understand other unmentioned technical effects according to the following description.

[0009] Technical Solutions

[0010] The image decoding method according to the present disclosure includes: collecting prediction information from neighboring blocks of a current block; deriving prediction parameters of the current block based on the collected prediction information; and deriving prediction information of each sub-block in the current block based on the prediction parameters.

[0011] The image encoding method according to the present disclosure includes: collecting prediction information from neighboring blocks of a current block; deriving prediction parameters of the current block based on the collected prediction information; and deriving prediction information of each subblock in the current block based on the prediction parameters.

[0012] In the image decoding / encoding method according to the present disclosure, when the prediction mode of the current block is intra prediction, the prediction information may be the intra prediction mode.

[0013] In the image decoding / encoding method according to the contents of the present disclosure, the prediction parameters include a pair of weights and offsets, and the intra-frame prediction mode of the sub-block can be obtained by summing the following values: a first value obtained by multiplying the x-axis position of the sub-block and the first weight, a second value obtained by multiplying the y-axis position of the sub-block and the second weight, and the offset.

[0014] In the image decoding / encoding method according to the present disclosure, when a result value obtained by summing the first value, the second value and the offset exceeds a range of a predefined intra prediction mode, the result value may be clipped to derive the intra prediction mode of the subblock.

[0015] In the image decoding / encoding method according to the contents of the present disclosure, when a result value obtained by summing a first value, a second value, and an offset exceeds a range of a predefined intra-frame prediction mode, the intra-frame prediction mode of the sub-block can be derived by adding a predefined constant to the result value or subtracting the predefined constant from the result value.

[0016] In the image decoding / encoding method according to the present disclosure, when the prediction mode of the current block is inter prediction, the prediction information may be a motion vector.

[0017] In the image decoding / encoding method according to the present disclosure, when a reference picture of a neighboring block is different from a base reference picture, scaled motion vectors of the neighboring block may be collected.

[0018] In the image decoding / encoding method according to the present disclosure, the base reference picture may be the first reference picture in the reference picture list, or may be a reference picture closest to the current picture in the reference picture list.

[0019] In the image decoding / encoding method according to the present disclosure, prediction parameters may be derived independently for each of the L0 direction and the L1 direction.

[0020] In the image decoding / encoding method according to the contents of the present disclosure, when inter-frame prediction based on an affine model is applied to the current block, a control point motion vector of the current block can be derived based on the prediction parameters, and a motion vector of each sub-block in the sub-block can be derived based on the control point motion vector.

[0021] The image decoding / encoding method according to the present disclosure may further include: determining whether to derive prediction parameters based on the collected prediction information. In this case, when it is determined not to derive prediction parameters based on the collected prediction information, prediction parameters may be derived from neighboring blocks adjacent to the current block.

[0022] In the image decoding / encoding method according to the present disclosure, the prediction parameters of the current block may be set to be the same as the prediction parameters of the neighboring blocks.

[0023] In the image decoding / encoding method according to the present disclosure, a neighboring block used by a current block to derive a prediction parameter may be specified based on index information.

[0024] A computer-readable recording medium storing a bit stream encoded by the image encoding method according to the present disclosure may be provided.

[0025] Technical Effects

[0026] According to the present disclosure, prediction information may be derived in units of sub-blocks based on prediction parameters, thereby improving encoding / decoding efficiency.

[0027] According to the present disclosure, prediction parameters of a current block may be derived by referring to pre-reconstructed neighboring blocks, thereby improving encoding / decoding efficiency.

[0028] Effects obtainable from the present disclosure are not limited to the above-mentioned effects, and other unmentioned effects can be clearly understood by a person of ordinary skill in the technical field to which the present disclosure pertains from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

[0030] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0031] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0032] Figure 4 and Figure 5 Examples of multiple intra prediction modes according to the present disclosure are shown.

[0033] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0034] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0035] Figure 8 A directional mode based intra prediction method according to the present disclosure is shown.

[0036] Fig. 9 A method for deriving samples at fractional positions is shown.

[0037] Fig.10 and Fig.11 It is shown that the tangent value of the angle is scaled by a factor of 32 in intra prediction mode.

[0038] Fig.12 is a diagram showing aspects of intra prediction when the directional mode is one of mode number 34 to mode number 49.

[0039] Fig.13 is a diagram for describing an example of generating top reference samples by interpolating left reference samples.

[0040] Fig.14 An example of performing intra prediction by using reference samples arranged in a 1D array is shown.

[0041] Fig.15 and Fig.16 It is a flowchart of the inter-frame prediction method.

[0042] Fig.17 An example of performing motion estimation is shown.

[0043] Fig.18 and Fig.19 An example of generating a prediction block of a current block based on motion information generated through motion estimation is shown.

[0044] Fig. 20 The locations referenced for deriving motion vector prediction values ​​are shown.

[0045] Fig.21 is a flowchart of a process of allocating prediction information to each subblock when intra prediction is applied to a current block.

[0046] Fig. 22 is a diagram for describing an example of collecting intra prediction modes around a current block.

[0047] Fig.23 is a flowchart of a process of allocating prediction information to each subblock when inter prediction is applied to a current block.

[0048] Fig.24 The control point motion vectors of the current block are shown. DETAILED DESCRIPTION

[0049] Since the present disclosure can be variously changed and has several embodiments, specific embodiments will be described in the drawings and described in detail. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood that the present disclosure includes all changes, equivalents or substitutes included within the conceptual and technical scope of the present disclosure. When describing each of the drawings, similar reference numerals are used for similar parts.

[0050] Terms such as first, second, etc. may be used to describe various components, but these components should not be limited by the terms. Terms are used only to distinguish one component from other components. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component, without exceeding the scope of the rights of the present disclosure. Terms and / or combinations of multiple related input items or any item of multiple related input items.

[0051] When a component is referred to as being "linked" or "connected" to another component, it should be understood that the component may be directly linked or connected to the other component, but other components may also exist in between. On the other hand, when a component is referred to as being "directly linked" or "directly connected" to another component, it should be understood that no other components exist in between.

[0052] Because the terms used in this application are only used to describe specific embodiments, they are not intended to limit the present disclosure. Singular expressions include plural expressions unless the singular expressions clearly have different meanings in the context. In this application, it should be understood that terms such as "including" or "having" refer to the presence of characteristics, numbers, steps, movements, parts, parts or combinations thereof entered in the specification, but do not exclude the possibility of pre-adding or existing one or more other characteristics, numbers, steps, movements, parts, parts or combinations thereof.

[0053] Hereinafter, with reference to the accompanying drawings, a desired embodiment of the present disclosure will be described in more detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated description of the same components is omitted.

[0054] Figure 1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

[0055] Reference Figure 1The image encoding device (100) may include: a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a rearrangement unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150) and a memory (155).

[0056] because Figure 1 Each of the structural units shown in is shown independently to represent different characteristic functions in the image encoding device, which does not mean that each structural unit is composed of separate hardware or a software unit. That is, since each structural unit is included by being listed as each structural unit for the convenience of description, at least two structural units in each structural unit can be combined to constitute one structural unit, or one structural unit can be divided into multiple structural units to perform functions, and even the integrated implementation and separate implementation of each structural unit are included in the scope of the rights of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0057] In addition, some components may be only optional components for improving performance, rather than essential components for performing the basic functions in the present disclosure. The present disclosure may be implemented by including only structural units necessary for realizing the essence of the present disclosure without including components only for improving performance, and a structure including only essential components without including optional components only for improving performance is also included in the scope of the rights of the present disclosure.

[0058] The picture segmentation unit (110) may segment an input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In the picture segmentation unit (110), a picture may be segmented into a combination of a plurality of decoding units, prediction units, and transform units, and the picture may be encoded by selecting a combination of a decoding unit, a prediction unit, and a transform unit according to a predetermined criterion (e.g., a cost function).

[0059] For example, one picture may be partitioned into a plurality of decoding units. In order to partition the decoding units in the picture, a recursive tree structure such as a quadtree may be used, and a decoding unit partitioned into other decoding units by using one image or a maximum decoding unit as a path may be partitioned with as many child nodes as the number of the partitioned decoding units. A decoding unit that is no longer partitioned according to a specific restriction becomes a leaf node. In other words, assuming that only square partitioning is possible for one decoding unit, one decoding unit may be partitioned into up to four other decoding units.

[0060] Hereinafter, in an embodiment of the present disclosure, a decoding unit may be used as a unit for encoding, or may be used as a unit for decoding.

[0061] The prediction unit may be partitioned with at least one square or rectangular shape, etc., in the same size in one decoding unit, or may be partitioned such that any one of the prediction units partitioned in one decoding unit may have a different shape and / or size from another prediction unit.

[0062] When a prediction unit for which intra prediction is performed based on a coding unit is generated and the prediction unit is not a minimum coding unit, intra prediction may be performed without partitioning the prediction unit into a plurality of prediction units N×N.

[0063] The prediction unit (120, 125) may include an inter-frame prediction unit (120) that performs inter-frame prediction and an intra-frame prediction unit (125) that performs intra-frame prediction. It may be determined whether to perform inter-frame prediction or intra-frame prediction on the decoding unit, and detailed information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, the processing unit that performs the prediction may be different from the processing unit that determines the prediction method and details. For example, the prediction method, prediction mode, etc. may be determined in the prediction unit, and the prediction may be performed in the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and may be sent to the decoding device. When a specific encoding mode is used, the original block may be encoded as is without generating a prediction block through the prediction unit (120, 125) and sent to the decoding unit.

[0064] The inter-frame prediction unit (120) may predict the prediction unit based on information about at least one picture in the previous picture or the subsequent picture of the current picture, or in some cases may predict the prediction unit based on information about some coding regions in the current picture. The inter-frame prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0065] The reference picture interpolation unit may receive reference picture information from a memory (155) and generate pixel information equal to or less than an integer pixel in the reference picture. For luma pixels, an 8-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 4 pixels. For chrominance signals, a 4-tap interpolation filter based on DCT with different filter coefficients may be used to generate pixel information equal to or less than an integer pixel in units of 1 / 8 pixels.

[0066] The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. As a method for calculating a motion vector, various methods such as FBMA (block matching algorithm based on full search), TSS (three-step search), NTS (new three-step search algorithm), etc. may be used. Based on the interpolated pixel, the motion vector may have a motion vector value in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit may predict the current prediction unit by changing the motion prediction method. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, an intra-block copy method, etc. may be used as a motion prediction method.

[0067] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When the neighboring block in the current prediction unit is a block on which inter prediction is performed and therefore the reference pixel is a pixel on which inter prediction is performed, the reference pixel included in the block on which inter prediction is performed can be used by replacing it with the reference pixel information of the surrounding blocks on which intra prediction is performed. In other words, when the reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one reference pixel among the available reference pixels.

[0068] The prediction mode of the intra prediction may have a directional prediction mode that uses reference pixel information according to a prediction direction when performing prediction and a non-directional mode that does not use directional information. The mode for predicting luminance information may be different from the mode for predicting chrominance information, and the chrominance information may be predicted using the intra prediction mode information for predicting luminance information or the predicted luminance signal information.

[0069] In the case where the size of the prediction unit is the same as the size of the transform unit when performing intra prediction, intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit. However, if the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0070] The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of AIS filter applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode in the current prediction unit may be predicted based on the intra prediction modes in the prediction units surrounding the current prediction unit. When the prediction mode in the current prediction unit is predicted by using the mode information predicted based on the surrounding prediction units, if the intra prediction mode in the current prediction unit is the same as the intra prediction mode in the surrounding prediction units, information about the prediction mode in the current prediction unit being the same as the prediction mode in the surrounding prediction units may be transmitted by using predetermined flag information, and if the prediction mode in the current prediction unit is different from the prediction mode in the surrounding prediction units, the prediction mode information of the current block may be encoded by performing entropy coding.

[0071] In addition, a residual block including information on a residual value that is a difference value between a prediction unit predicted based on the prediction unit generated in the prediction unit (120, 125) and an original block in the prediction unit may be generated. The generated residual block may be input to the transform unit (130).

[0072] The transform unit (130) can transform the original block and the residual block described below by using a transform method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT, and the residual block includes residual value information in the prediction unit generated by the prediction unit (120, 125). Whether to apply DCT, DST or KLT to transform the residual block can be determined based on intra-frame prediction mode information in the prediction unit used to generate the residual block.

[0073] The quantization unit (135) may quantize the value transformed into the frequency domain in the transformation unit (130). The quantization coefficient may be changed according to the importance or block of the image. The value calculated in the quantization unit (135) may be provided to the inverse quantization unit (140) and the rearrangement unit (160).

[0074] The rearrangement unit 160 may perform rearrangement on coefficient values ​​of the quantized residual value.

[0075] The rearrangement unit (160) can change the coefficients in the shape of a two-dimensional block into the shape of a one-dimensional vector by a coefficient scanning method. For example, the rearrangement unit (160) can scan from the DC coefficient to the coefficients in the high frequency domain by using a zigzag scanning method and change them into the shape of a one-dimensional vector. Depending on the size of the transform unit and the intra-frame prediction mode, instead of the zigzag scanning, vertical scanning of scanning the coefficients in the shape of a two-dimensional block along the column direction or horizontal scanning of scanning the coefficients in the shape of a two-dimensional block along the row direction can be used. In other words, which scanning method of zigzag scanning, vertical scanning, and horizontal scanning to be used can be determined according to the size of the transform unit and the intra-frame prediction mode.

[0076] The entropy coding unit (165) may perform entropy coding based on the value calculated by the rearrangement unit (160). For example, the entropy coding may use various coding methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding).

[0077] The entropy coding unit (165) can encode various information from the rearrangement unit (160) and the prediction unit (120, 125), such as residual value coefficient information and block type information in the decoding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0078] The entropy encoding unit (165) may perform entropy encoding on the coefficient values ​​in the decoding unit input from the rearrangement unit (160).

[0079] The inverse quantization unit (140) and the inverse transform unit (145) inversely quantize the value quantized in the quantization unit (135) and inversely transform the value transformed in the transform unit (130). The residual value generated by the inverse quantization unit (140) and the inverse transform unit (145) can be combined with the prediction unit predicted by the motion prediction unit, the motion compensation unit, and the intra-frame prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

[0080] The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0081] The deblocking filter can remove block distortion generated by the boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it can be determined whether to apply the deblocking filter to the current block based on the pixels included in several rows or columns included in the block. When the deblocking filter is applied to the block, a strong filter or a weak filter can be applied according to the required deblocking filter strength. In addition, when applying the deblocking filter, when performing horizontal filtering and vertical filtering, horizontal directional filtering and vertical directional filtering can be set to parallel processing.

[0082] The offset correction unit can correct the offset from the original image in units of pixels for the image to be deblocked. In order to perform offset correction on a specific picture, the area where the offset is to be performed can be determined after dividing the pixels included in the image into a certain number of areas, and a method of applying the offset to the corresponding area or a method of applying the offset by considering edge information of each pixel can be used.

[0083] Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing a filtered reconstructed image with an original image. After the pixels included in the image are divided into predetermined groups, filtering may be performed distinguishably by group by determining a filter to be applied to the corresponding group. Information on whether ALF is applied may be transmitted per coding unit (CU) for a luminance signal, and the shape and filter coefficient of the ALF filter to be applied may be different for each block. In addition, an ALF filter having the same shape (fixed shape) may be applied regardless of the characteristics of the block to be applied.

[0084] The memory (155) may store a reconstructed block or picture calculated by the filter unit (150), and when inter-frame prediction is performed, the stored reconstructed block or picture may be provided to the prediction unit (120, 125).

[0085] Figure 2 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0086] Reference Figure 2 , the image decoding device (200) may include: an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transform unit (225), a prediction unit (230, 235), a filter unit (240) and a memory (245).

[0087] When an image bitstream is input from the image encoding device, the input bitstream may be decoded according to a process reverse to that of the image encoding device.

[0088] The entropy decoding unit (210) can perform entropy decoding according to a process opposite to the process of performing entropy encoding in the entropy encoding unit of the image encoding device. For example, various methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), CABAC (Context Adaptive Binary Arithmetic Coding) can be applied in response to the method performed in the image encoding device.

[0089] The entropy decoding unit (210) can decode information related to intra-frame prediction and inter-frame prediction performed in the encoding device.

[0090] The rearrangement unit (215) may perform rearrangement based on a method of rearranging the bit stream entropy decoded in the entropy decoding unit (210) in the coding unit. The coefficients expressed in the form of a one-dimensional vector may be rearranged by reconstructing the coefficients in the form of a two-dimensional block. The rearrangement unit (215) may receive information related to coefficient scanning performed in the coding unit, and perform rearrangement by performing scanning in reverse based on a scanning order performed in the corresponding coding unit.

[0091] The inverse quantization unit (220) may perform inverse quantization based on the quantization parameter provided from the encoding device and the coefficient value of the rearranged block.

[0092] The inverse transform unit (225) can perform the transform performed in the transform unit, that is, the inverse transform for DCT, DST and KLT (that is, inverse DCT, inverse DST and inverse KLT) on the result of the quantization performed in the image encoding device. The inverse transform can be performed based on the transmission unit determined in the image encoding device. In the inverse transform unit (225) of the image decoding device, the transform technology (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of information such as a prediction method, a size of a current block, a prediction mode, etc.

[0093] The prediction unit (230, 235) may generate a prediction block based on information related to generation of the prediction block provided from the entropy decoding unit (210) and pre-decoded block or picture information provided from the memory (245).

[0094] As described above, when the size of the prediction unit is the same as the size of the transform unit when performing intra prediction in the same manner as the operation in the image encoding device, intra prediction of the prediction unit can be performed based on the pixel at the left position, the pixel at the upper left position, and the pixel at the top position of the prediction unit, but when the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed by using reference pixels based on the transform unit. In addition, intra prediction using N×N partitioning can be used only for the minimum decoding unit.

[0095] The prediction unit (230, 235) may include: a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information such as prediction unit information, prediction mode information of an intra-frame prediction method, motion prediction related information of an inter-frame prediction method, etc. input from the entropy decoding unit (210), divide the prediction unit in the current decoding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit (230) may perform inter-frame prediction on the current prediction unit based on information included in at least one of a previous picture or a subsequent picture of a current picture including the current prediction unit by using information required for inter-frame prediction in the current prediction unit provided from the image encoding device. Alternatively, inter-frame prediction may be performed based on information about some areas pre-reconstructed in the current picture including the current prediction unit.

[0096] In order to perform inter prediction, whether a motion prediction method in a prediction unit included in a corresponding coding unit is a skip mode, a merge mode, an AMVP mode, or an intra block copy mode may be determined on a coding unit basis.

[0097] The intra prediction unit (235) can generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that performs intra prediction, intra prediction can be performed based on intra prediction mode information in the prediction unit provided from the image encoding device. The intra prediction unit (235) may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As a part of performing filtering on the reference pixels of the current block, the AIS filter can be applied by determining whether to apply the filter according to the prediction mode in the current prediction unit. By using the prediction mode and AIS filter information in the prediction unit provided from the image encoding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0098] When the prediction mode in the prediction unit is a prediction unit that performs intra-frame prediction based on a pixel value interpolated from a reference pixel, the reference pixel interpolation unit may interpolate the reference pixel so as to generate the reference pixel in units of pixels equal to or less than an integer value. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. When the prediction mode of the current block is a DC mode, the DC filter may generate a prediction block by filtering.

[0099] The reconstructed block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0100] Information about whether to apply a deblocking filter to a corresponding block or picture and information about whether to apply a strong filter or a weak filter when applying the deblocking filter can be provided from the image encoding device. Information related to the deblocking filter provided from the image encoding device can be provided in the deblocking filter of the image decoding device, and deblocking filtering of the corresponding block can be performed in the image decoding device.

[0101] The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image when encoding is performed, offset value information, and the like.

[0102] ALF may be applied to the decoding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoding device. Such ALF information may be provided by being included in a specific parameter set.

[0103] The memory (245) may store the reconstructed picture or block to be used as a reference picture or a reference block, and provide the reconstructed picture to the output unit.

[0104] As described above, hereinafter, in the embodiments of the present disclosure, for convenience of description, a decoding unit is used as a term of a decoding unit, but may be a unit that performs decoding as well as encoding.

[0105] In addition, since the current block represents a block to be encoded / decoded, it may represent a decoding tree block (or decoding tree unit), a decoding block (or decoding unit), a transform block (or transform unit), or a prediction block (or prediction unit), etc., according to the encoding / decoding step. In the present specification, a "unit" may represent a basic unit for performing a specific encoding / decoding process, and a "block" may represent a pixel array of a predetermined size. Unless otherwise classified, "block" and "unit" may be used interchangeably. For example, in the embodiments described later, it may be understood that a decoding block (decoding block) and a decoding unit (decoding unit) may be used interchangeably.

[0106] Figure 3 An image encoding / decoding method performed by the image encoding / decoding device according to the present disclosure is shown.

[0107] Reference Figure 3 , a reference line for intra prediction of a current block may be determined ( S300 ).

[0108] The current block may use one or more reference line candidates among a plurality of reference line candidates predefined in an image encoding / decoding device as a reference line for intra prediction. Here, the plurality of predefined reference line candidates may include adjacent reference lines adjacent to the current block to be decoded and N non-adjacent reference lines that are 1 sample to N samples away from the boundary of the current block. N may be an integer of 1, 2, 3 or more. In the following, for ease of description, it is assumed that the plurality of reference line candidates available for the current block include adjacent reference line candidates and three non-adjacent reference line candidates, but is not limited thereto. In other words, of course, the plurality of reference line candidates available for the current block may include four or more non-adjacent reference line candidates.

[0109] The image encoding device may determine the best reference line candidate among multiple reference line candidates and encode an index for specifying the best reference line candidate. The image decoding device may determine the reference line of the current block based on the index signaled by the bitstream. The index may specify any one of the multiple reference line candidates. The reference line candidate specified by the index may be used as the reference line of the current block.

[0110] The number of indexes signaled to determine the reference line of the current block may be 1, 2, or more. As an example, when the number of indexes signaled above is 1, the current block may perform intra-frame prediction by using only a single reference line candidate specified by the signaled index among multiple reference line candidates. Alternatively, when the number of indexes signaled above is at least 2, the current block may perform intra-frame prediction by using multiple reference line candidates specified by multiple indexes among multiple reference line candidates.

[0111] Reference Figure 3 , the intra prediction mode of the current block can be determined (S310).

[0112] The intra prediction mode of the current block can be determined among a plurality of intra prediction modes predefined in the image encoding / decoding device. Figure 4 and Figure 5 To describe multiple predefined intra prediction modes.

[0113] Figure 4 Examples of multiple intra prediction modes according to the present disclosure are shown.

[0114] Reference Figure 4 , the plurality of intra prediction modes predefined in the image encoding / decoding device may include a non-directional mode and a directional mode. The non-directional mode may include at least one of a planar mode or a DC mode. The directional mode may include directional modes 2 to 66.

[0115] Directional mode can be compared Figure 4 The directional pattern shown is much more extended. Figure 5 An example in which the directional mode is extended is shown.

[0116] exist Figure 5 , adding mode -1 to mode -14 and mode 67 to mode 80 are shown. These directional modes may be referred to as wide-angle intra prediction modes. Whether to use the wide-angle intra prediction mode may be determined based on the shape of the current block. As an example, if the current block is a non-square block having a width greater than a height, some directional modes (e.g., numbers 2 to 15) may be transformed into wide-angle intra prediction modes between numbers 67 and 80. On the other hand, if the current block is a non-square block having a height greater than a width, some directional modes (e.g., numbers 53 to 66) may be transformed into wide-angle intra prediction modes between numbers -1 and -14.

[0117] The range of available wide-angle intra prediction modes may be adaptively determined according to the ratio of the width to the height of the current block. Table 1 shows the range of available wide-angle intra prediction modes according to the ratio of the width to the height of the current block.

[0118] [Table 1]

[0119] Width / Height Available wide-angle intra prediction mode range W / H=16 67～80 W / H=8 67～78 W / H=4 67～76 W / H=2 67～74 W / H=1 none W / H=1 / 2 -1～-8 W / H=1 / 4 -1～-10 W / H=1 / 8 -1～-12 W / H=1 / 16 -1～-14

[0120] K candidate modes (most probable modes, MPM) may be selected from among a plurality of intra prediction modes. A candidate list including the selected candidate modes may be generated. An index indicating any one of the candidate modes belonging to the candidate list may be signaled. The intra prediction mode of the current block may be determined based on the candidate mode indicated by the index. As an example, the candidate mode indicated by the index may be configured as the intra prediction mode of the current block. Alternatively, the intra prediction mode of the current block may be determined based on the value of the candidate mode indicated by the index and a predetermined difference. The difference may be defined as the difference between the value of the intra prediction mode of the current block and the value of the candidate mode indicated by the index. The difference may be signaled by a bitstream. Alternatively, the difference may be a value predefined in an image encoding / decoding device. Alternatively, the intra prediction mode of the current block may be determined based on a flag showing whether there is a mode identical to the intra prediction mode of the current block in the candidate list. As an example, when the flag is a first value, the intra prediction mode of the current block may be determined from the candidate list. In this case, an index indicating any one of the multiple candidate modes belonging to the candidate list may be signaled. The candidate mode indicated by the index may be configured as the intra prediction mode of the current block. On the other hand, when the flag is a second value, any one of the remaining intra prediction modes may be configured as the intra prediction mode of the current block. The remaining intra prediction modes may refer to a mode excluding the candidate mode belonging to the candidate list in a plurality of predefined intra prediction modes. When the flag is a second value, an index indicating any one of the remaining intra prediction modes may be signaled. The intra prediction mode indicated by the signaled index may be configured as the intra prediction mode of the current block. The intra prediction mode of a chroma block may be selected from the intra prediction mode candidates of a plurality of chroma blocks. To this end, index information indicating one of the intra prediction mode candidates of the intra prediction mode of the chroma block may be explicitly encoded and signaled by a bitstream. Table 2 shows the intra prediction mode candidates of the chroma block.

[0121] [Table 2]

[0122]

[0123] In the example of Table 2, direct mode (DM) means setting the intra-frame prediction mode of the luminance block present at the same position as the chrominance block to the intra-frame prediction mode of the chrominance block. As an example, if the intra-frame prediction mode of the luminance block (luminance mode) is number 0 (planar mode) and the index indicates number 2, the intra-frame prediction mode of the chrominance block can be determined as the horizontal mode (number 18). As an example, if the intra-frame prediction mode of the luminance block (luminance mode) is number 1 (DC mode) and the index indicates number 0, the intra-frame prediction mode of the chrominance block can be determined as the planar mode (number 0). Therefore, the intra-frame prediction mode of the chrominance block can also be set to Figure 4 or Figure 5 One of the intra prediction modes shown. The intra prediction mode of the current block may be used to determine the reference line of the current block, and in this case, S310 may be performed before S300.

[0124] Reference Figure 3 , intra prediction may be performed on the current block based on the intra prediction mode and the reference line of the current block (S320).

[0125] In the following, by reference Figures 6 to 8 The intra prediction method of each intra prediction mode is described in detail. However, for ease of description, it is assumed that a single reference line is used for intra prediction of the current block, but even in the case of using multiple reference lines, the intra prediction method described later can be applied in the same / similar manner.

[0126] Figure 6 A planar mode based intra prediction method according to the present disclosure is shown.

[0127] Reference Figure 6 , T represents a reference sample located at the upper right corner of the current block, and L represents a reference sample located at the lower left corner of the current block. P1 can be generated by interpolation in the horizontal direction. As an example, P1 can be generated by interpolating T and a reference sample located on the same horizontal line as P1. P2 can be generated by interpolation in the vertical direction. As an example, P2 can be generated by interpolating L and a reference sample located on the same vertical line as P2. As in the following formula 1, the current sample in the current block can be predicted by the weighted sum of P1 and P2.

[0128] [Formula 1]

[0129] (α×P1+β×P2) / (α+β)

[0130] In Formula 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β may have the same value or different values. If the width and height of the current block are the same, weights α and β may be configured to be the same, and the predicted sample of the current sample may be configured as the average of P1 and P2. When the width and height of the current block are different, weights α and β may have different values. As an example, when the width is greater than the height, a smaller value may be configured as a weight corresponding to the width of the current block, and a larger value may be configured as a weight corresponding to the height of the current block. Conversely, when the width is greater than the height, a larger value may be configured as a weight corresponding to the width of the current block, and a smaller value may be configured as a weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block may refer to β, and the weight corresponding to the height of the current block may refer to α.

[0131] Figure 7 A DC mode based intra prediction method according to the present disclosure is shown.

[0132] Reference Figure 7 , the average value of the surrounding samples adjacent to the current block can be calculated, and the calculated average value can be configured as the prediction value of all samples in the current block. Here, the surrounding samples may include the top reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value may be calculated by using only the top reference sample or the left reference sample. As an example, when the width of the current block is greater than the height, the average value may be calculated by using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated by using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value may be calculated by using only the top reference sample of the current block. On the other hand, when the width of the current block is less than the height, the average value may be calculated by using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value may be calculated by using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value may be calculated by using only left reference samples of the current block.

[0133] Figure 8 A directional mode based intra prediction method according to the present disclosure is shown.

[0134] When the intra-frame prediction mode of the current block is a directional mode, projection can be performed to the reference line according to the angle of the corresponding directional mode. When there is a reference sample at the projection position, the corresponding reference sample can be configured as a prediction sample of the current sample. If there is no reference sample at the projection position, a sample corresponding to the projection position can be generated by using one or more surrounding samples adjacent to the projection position. As an example, interpolation can be performed based on two or more surrounding samples adjacent in two directions relative to the projection position to generate a sample corresponding to the projection position. Alternatively, a surrounding sample adjacent to the projection position can be configured as a sample corresponding to the projection position. In this case, among the multiple surrounding samples adjacent to the projection position, the surrounding sample closest to the projection position can be used. The sample corresponding to the projection position can be configured as a prediction sample of the current sample.

[0135] Reference Figure 8 , for the current sample B, when projection is performed on the reference line at the corresponding position according to the angle of the intra-frame prediction mode, there is a reference sample at the projection position (i.e., the reference sample R3 at the integer position). In this case, the reference sample at the projection position can be configured as a prediction sample for the current sample B. For the current sample A, when projection is performed on the reference line at the corresponding position according to the angle of the intra-frame prediction mode, there is no reference sample at the projection position (i.e., the reference sample at the integer position). In this case, interpolation can be performed based on surrounding samples (e.g., R2 and R3) adjacent to the projection position to generate a sample (r) at a fractional position. The generated sample (r) at the fractional position can be configured as a prediction sample for the current sample A.

[0136] Fig. 9 A method for deriving samples at fractional positions is shown.

[0137] exist Fig. 9 In the example of , variable h refers to the vertical distance (i.e., vertical distance) from the position of prediction sample A to the reference sample line, and variable w refers to the horizontal distance (i.e., horizontal distance) from the position of prediction sample A to the fractional position sample. In addition, variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and variable x refers to the fractional position.

[0138] The variable w can be obtained as shown in Equation 2 below.

[0139] [Formula 2]

[0140] w=tanθ

[0141] Then, if we remove the integer positions from the variable w, we can finally get the fractional positions.

[0142] The fractional position samples may be generated by interpolating adjacent integer position reference samples. As an example, the fractional position reference samples at the x position may be generated by interpolating integer position reference samples R2 and integer position reference samples R3.

[0143] When deriving fractional position samples, a scaling factor can be used to avoid real number operations. As an example, when the scaling factor f is set to 32, such as Figure 8 In the example shown in (b), the distance between adjacent integer reference samples can be set to 32 instead of 1.

[0144] In addition, the tangent value of the angle θ determined according to the directionality of the intra prediction mode may also be scaled by using the same scaling factor (eg, 32).

[0145] Fig.10 and Fig.11 It is shown that the tangent value of the angle is scaled by a factor of 32 in intra prediction mode.

[0146] Fig.10 shows the scaling results of the tangent value for non-wide angle intra prediction mode, and Fig.11 The scaling results of the tangent values ​​for the wide angle intra prediction mode are shown.

[0147] If the tangent value (tanθ) of the angle value used for the intra prediction mode is positive, intra prediction can be performed by using only one of the reference samples belonging to the top line of the current block (i.e., top reference samples) or the reference samples belonging to the left line of the current block (i.e., left reference samples). Meanwhile, when the tangent value of the angle value used for the intra prediction mode is negative, both the reference samples at the top position and the reference samples at the left position are used.

[0148] In this case, to simplify the implementation, the left reference samples may be projected to the top, or may be projected to the left to arrange the reference samples in a 1D array form, and intra prediction may be performed using the reference samples in the 1D array form.

[0149] Fig.12 is a diagram showing aspects of intra prediction when the directional mode is one of mode number 34 to mode number 49.

[0150] When the intra prediction mode of the current block is one of mode number 34 to mode number 49, intra prediction is performed by using not only the top reference sample of the current block but also the left reference sample. Fig.12 In the example shown, the reference samples of the top line may be generated by copying the reference samples at the left position of the current block to the position of the top line or by interpolating the reference samples at the left position.

[0151] As an example, when a reference sample of position A on the top of the current block is obtained, projection may be performed at position A on the top line to the left line of the current block in consideration of the directionality of the intra-prediction mode of the current block. If the projection position is referred to as a, the value corresponding to position a may be copied, or a fractional position value corresponding to a may be generated and set to the value of position A. As an example, if position a is an integer position, the value of position A may be generated by copying the integer position reference sample. On the other hand, when position a is a fractional position, the reference sample at the top position of position a and the reference sample at the bottom position of position a may be interpolated, and the interpolated value may be set to the value of position A. Meanwhile, at position A on the top of the current block, the direction of the projection to the left line of the current block may be parallel and opposite to the direction of the intra-prediction mode of the current block.

[0152] Fig.13 is a diagram for describing an example of generating top reference samples by interpolating left reference samples.

[0153] exist Fig.13 , variable h represents the horizontal distance between position A on the top line and position a on the left line. Variable w represents the vertical distance between position A on the top line and position a on the left line. In addition, variable θ refers to an angle predefined according to the directionality of the intra prediction mode, and variable x refers to a fractional position.

[0154] The variable h can be obtained as shown in Equation 3 below.

[0155] [Formula 3]

[0156] h=w / tanθ

[0157] Afterwards, if the integer positions are removed from the variable h, the fractional positions can be finally obtained.

[0158] When deriving fractional position samples, scaling factors can be used to avoid real number operations. As an example, the tangent of the variable θ can be scaled by using a scaling factor f1. Here, since the direction of the projection to the left line is parallel and opposite to the directional prediction mode, we can use Fig.10 and Fig.11 Scaled tangent values ​​shown.

[0159] When the scaling factor f1 is applied, Equation 3 may be modified and used as in Equation 4 below.

[0160] [Formula 4]

[0161] h=(w*f1*f2) / (f1*tanθ)

[0162] In the same manner as above, a 1D reference sample array may be configured with only reference samples belonging to the top line. Thus, intra prediction of the current block may be performed by using only the top reference samples comprising the 1D array.

[0163] Fig.14 An example of performing intra prediction by using reference samples arranged in a 1D array is shown.

[0164] like Fig.14 In the example shown, the left reference samples may be projected to generate the top reference samples, and thus, the prediction samples of the current block may be obtained by using only the reference samples belonging to the top line.

[0165] and Fig.12 and Fig.14 In contrast to what is shown in , the top reference sample can be projected to the left line to configure the 1D reference sample array with only the reference samples belonging to the left line. Specifically, for pattern numbers 19 to 33 of the directional patterns in which the tangent value (tanθ) of the angle of the directional pattern is negative, the reference samples belonging to the top line can be projected to the left line to generate the left reference samples.

[0166] When encoding the current picture, redundant data between pictures can be removed by inter-frame prediction. Inter-frame prediction can be performed in units of blocks. Specifically, a prediction block of the current block can be generated from a reference picture by using motion information of the current block. Here, the motion information may include at least one of a motion vector, a reference picture index, and a prediction direction.

[0167] Fig.15 and Fig.16 It is a flow chart of the inter-frame prediction method.

[0168] Fig.15 represents the operation of the encoder, and Fig.16 Represents the operation of the decoder.

[0169] In the encoder, motion estimation is performed (S1510), and a prediction block is obtained based on motion information derived from the motion estimation result (S1520). Here, the motion information may include at least one of a motion vector, a reference picture index, a motion vector accuracy, a bidirectional weight, whether to perform L0 prediction, or whether to perform L1 prediction.

[0170] Also, in the encoder, an inter prediction mode for performing inter prediction may be determined, and information for deriving motion information may be encoded according to the determined inter prediction mode (S1530).

[0171] In the decoder, the inter-frame prediction mode can be determined based on the information signaled from the bitstream (S1610), and motion information can be obtained according to the determined inter-frame prediction mode (S1620). Once the motion information is obtained, a prediction block of the current block can be obtained based on the obtained motion information (S1630).

[0172] Hereinafter, each step will be described in detail.

[0173] The motion information of the current block may be generated through motion estimation.

[0174] Fig.17 An example of performing motion estimation is shown.

[0175] exist Fig.17 In , it is assumed that the picture order count (POC) of the current picture is T, and the POC of the reference picture is (T-1).

[0176] The search range for motion estimation may be set from the same position as the reference point of the current block in the reference picture. Here, the reference point may be the position of the upper left sample of the current block.

[0177] As an example, in Fig.17 , it is shown that based on the reference point, quadrilaterals of sizes (w0+w01) and (h0+h1) are set as the search range. In the example, w0, w1, h0, and h1 may have the same value. Alternatively, at least one of w0, w1, h0, and h1 may be set to have a value different from the others. Alternatively, the sizes of w0, w1, h0, and h1 may be determined not to exceed a coding tree unit (CTU) boundary, a slice boundary, a tile boundary, or a picture boundary.

[0178] Within the search range, reference blocks of the same size as the current block can be set, and then the cost of each reference block relative to the current block can be measured. The cost can be calculated by using the similarity between the two blocks.

[0179] As an example, the cost may be calculated based on the sum of absolute differences (SAD) between the original samples in the current block and the original samples (or reconstructed samples) in the reference block. As the SAD is smaller, the cost may be reduced.

[0180] Afterwards, after comparing the cost of each reference block, the reference block with the best cost can be set as the prediction block of the current block.

[0181] Then, the distance between the current block and the reference block can be set as a motion vector. Specifically, the x-coordinate difference and the y-coordinate difference between the current block and the reference block can be set as a motion vector.

[0182] Furthermore, an index of a picture including a reference block specified by motion estimation is set as a reference picture index.

[0183] In addition, the prediction direction may be set based on whether the reference picture belongs to the L0 reference picture list or the L1 reference picture list.

[0184] In addition, motion estimation may be performed for each of the L0 direction and the L1 direction. When prediction is performed for both the L0 direction and the L1 direction, motion information in the L0 direction and motion information in the L1 direction may be generated, respectively.

[0185] Fig.18 and Fig.19 An example of generating a prediction block of a current block based on motion information generated through motion estimation is shown.

[0186] Fig.18 An example of generating a prediction block by unidirectional (ie, L0 direction) prediction is shown, and Fig.19 An example of generating a prediction block by bidirectional (ie, L0 direction and L1 direction) prediction is shown.

[0187] For unidirectional prediction, a prediction block of a current block is generated by using one motion information. As an example, the motion information may include prediction direction information indicating an L0 motion vector, an L0 reference picture index, and an L0 direction.

[0188] For bidirectional prediction, a prediction block is generated by using two motion information. As an example, a reference block in the L0 direction specified based on motion information in the L0 direction (L0 motion information) may be set as an L0 prediction block, and a reference block in the L1 direction specified based on motion information in the L1 direction (L1 motion information) may be generated as an L1 prediction block. Thereafter, the L0 prediction block and the L1 prediction block may be weighted to generate a prediction block of the current block.

[0189] exist Figures 17 to 19 In the example shown, it is shown that the L0 reference picture exists in the previous direction of the current picture (ie, the POC value is smaller than the POC value of the current picture), and the L1 reference picture exists in the subsequent direction of the current picture (ie, the POC value is greater than the POC value of the current picture).

[0190] However, unlike the example shown, the L0 reference picture may exist in the subsequent direction of the current picture, or the L1 reference picture may exist in the previous direction of the current picture. As an example, both the L0 reference picture and the L1 reference picture may exist in the previous direction of the current picture, or may exist in the subsequent direction of the current picture. Alternatively, bidirectional prediction may be performed by using the L0 reference picture existing in the subsequent direction of the current picture and the L1 reference picture existing in the previous direction of the current picture.

[0191] The motion information of the block for performing inter-frame prediction may be stored in a memory. In this case, the motion information may be stored in units of samples. Specifically, the motion information of the block to which a specific sample belongs may be stored as the motion information of the specific sample. The stored motion information may be used to derive the motion information of a neighboring block to be encoded / decoded in the future.

[0192] In the encoder, information on encoding residual samples corresponding to the difference between the samples of the current block (ie, the original samples) and the predicted samples and motion information required to generate the predicted block may be signaled to the decoder. In the decoder, information on the signaled difference may be decoded to obtain the difference samples, and the predicted samples in the predicted block generated by using the motion information may be added to the difference samples to generate the reconstructed samples.

[0193] In this case, in order to effectively compress the motion information signaled to the decoder, one of a plurality of inter prediction modes may be selected. Here, the plurality of inter prediction modes may include a motion information merge mode and a motion vector prediction mode.

[0194] The motion vector prediction mode is a mode in which the difference between the motion vector and the motion vector prediction value is encoded and signaled. Here, the motion vector prediction value may be derived based on motion information of neighboring blocks or neighboring samples adjacent to the current block.

[0195] Fig. 20 The locations referenced for deriving motion vector prediction values ​​are shown.

[0196] For convenience of description, it is assumed that the current block has a size of 4×4.

[0197] In the example shown, "LB" represents the samples included in the bottom row of the leftmost column in the current block. "RT" represents the samples included in the top row of the rightmost column in the current block. A0 to A4 represent samples adjacent to the left side of the current block, and B0 to B5 represent samples adjacent to the top of the current block. As an example, A1 represents a sample adjacent to the left side of LB, and B1 represents a sample adjacent to the top of RT. A neighboring block including a sample that is spatially adjacent to the current block (i.e., one of A0 to A4 or one of B0 to B5) may be referred to as a spatially adjacent block.

[0198] Col represents the position of the sample adjacent to the lower right corner of the current block in the co-located picture. The co-located picture is a picture different from the current picture, and the information for specifying the co-located picture can be explicitly encoded and signaled in the bitstream. Alternatively, a reference picture with a predefined reference picture index can be set as the co-located picture. A neighboring block including samples (i.e., Col) that are temporally adjacent to the current block can be referred to as a temporally adjacent block.

[0199] The motion vector prediction value of the current block may be derived from at least one motion vector prediction candidate included in the motion vector prediction list.

[0200] The number of motion vector prediction candidates that can be inserted into the motion vector prediction list (ie, the size of the list) may be predefined in the encoder and the decoder. As an example, the maximum number of motion vector prediction candidates may be 2.

[0201] The motion vector stored at the position of the neighboring sample adjacent to the current block or the scaled motion vector obtained by scaling the motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. In this case, the neighboring samples adjacent to the current block can be scanned in a predefined order to obtain the motion vector prediction candidate.

[0202] As an example, it may be confirmed whether the motion vector is stored at each position in the order of A0 to A4. And, according to the scanning order, the first found available motion vector may be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0203] As another example, it is confirmed whether the motion vector is stored at each position in the order of A0 to A4, and the motion vector at the position with the same reference picture as the current block found first can be inserted into the motion vector prediction list as a motion vector prediction candidate. If there is no neighboring sample with the same reference picture as the current block, the motion vector prediction candidate can be derived based on the first available vector found. Specifically, after scaling the first available motion vector found, the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. In this case, scaling can be performed based on the output order difference (i.e., POC difference) between the current picture and the reference picture and the output order difference (i.e., POC difference) between the current picture and the reference picture of the neighboring sample.

[0204] In addition, in the order of B0 to B5, it can be confirmed whether a motion vector is stored at each position. And, according to the scanning order, the first found available motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate.

[0205] As another example, it is confirmed whether the motion vector is stored at each position in the order of B0 to B5, and the motion vector at the position with the same reference picture as the current block found first can be inserted into the motion vector prediction list as a motion vector prediction candidate. If there is no neighboring sample with the same reference picture as the current block, the motion vector prediction candidate can be derived based on the first available vector found. Specifically, after scaling the first available motion vector found, the scaled motion vector can be inserted into the motion vector prediction list as a motion vector prediction candidate. In this case, scaling can be performed based on the output order difference (i.e., POC difference) between the current picture and the reference picture and the output order difference (i.e., POC difference) between the current picture and the reference picture of the neighboring sample.

[0206] When motion vectors are stored in units of blocks (eg, 4×4), a motion vector prediction candidate may be derived based on a motion vector of a block including a sample at a predetermined position.

[0207] As in the above-described example, motion vector prediction candidates may be derived from samples adjacent to the left side of the current block, and motion vector prediction candidates may be derived from samples adjacent to the top of the current block.

[0208] In this case, the motion vector prediction candidate derived from the left sample may be inserted into the motion vector prediction list before the motion vector prediction candidate derived from the top sample. In this case, the index assigned to the motion vector prediction candidate derived from the left sample may have a smaller value than the motion vector prediction candidate derived from the top sample.

[0209] Conversely, a motion vector prediction candidate derived from a top sample may be inserted into the motion vector prediction list prior to a motion vector prediction candidate derived from a left sample.

[0210] Among the motion vector prediction candidates included in the motion vector prediction list, the motion vector prediction candidate with the highest encoding efficiency may be set as the motion vector predictor (MVP) of the current block. And, index information indicating the motion vector prediction candidate set as the motion vector prediction value of the current block among the multiple motion vector prediction candidates may be encoded and signaled to the decoder. When the number of motion vector prediction candidates is 2, the index information may be a 1-bit flag (e.g., MVP flag). In addition, a motion vector difference (MVD) which is the difference between the motion vector of the current block and the motion vector predictor may be encoded and signaled to the decoder.

[0211] The decoder can configure the motion vector prediction list in the same manner as the encoder. In addition, index information can be decoded from the bitstream, and one of the multiple motion vector prediction candidates can be selected based on the decoded index information. The selected motion vector prediction candidate can be set as the motion vector prediction value of the current block.

[0212] Alternatively, the motion vector difference may be decoded from the bitstream. Afterwards, the motion vector of the current block may be derived by combining the motion vector prediction value and the motion vector difference.

[0213] When bidirectional prediction is applied to the current block, a motion vector prediction list may be generated for each of the L0 direction and the L1 direction. In other words, the motion vector prediction list may be composed of motion vectors in the same direction. Therefore, the motion vector of the current block and the motion vector prediction candidates included in the motion vector prediction list have the same direction.

[0214] When the motion vector prediction mode is selected, the reference picture index and the prediction direction information may be explicitly encoded and signaled to the decoder. As an example, when there are multiple reference pictures on the reference picture list and motion estimation is performed for each of the multiple reference pictures, the reference picture index for specifying the reference picture for deriving the motion information of the current block among the multiple reference pictures may be explicitly encoded and signaled to the decoder.

[0215] In this case, when only one reference picture is included in the reference picture list, encoding / decoding of the reference picture index may be omitted.

[0216] The prediction direction information may be an index indicating one of L0 unidirectional prediction, L1 unidirectional prediction, or bidirectional prediction. Alternatively, an L0 flag showing whether prediction of the L0 direction is performed and an L1 flag showing whether prediction of the L1 direction is performed may be encoded and signaled separately.

[0217] The motion information merge mode is a mode in which the motion information of the current block is set to be the same as the motion information of the neighboring block. In the motion information merge mode, the motion information may be decoded / encoded by using the motion information merge list.

[0218] The motion information merge candidate may be derived based on the motion information of the neighboring blocks or neighboring samples adjacent to the current block. As an example, after predefining the reference position around the current block, it may be confirmed whether there is motion information at the predefined reference position. When there is motion information at the predefined reference position, the motion information at the corresponding position may be inserted into the motion information merge list as a motion information merge candidate.

[0219] exist Figure 8In the example of , the predefined reference position may include at least one of A0, A1, B0, B1, B5, and Col. In addition, the motion information merge candidate may be derived in the order of A1, B1, B0, A0, B5, and Col.

[0220] When motion information is stored in units of blocks (eg, 4×4), a motion information merge candidate may be derived based on motion information of a block including samples at a predefined reference position.

[0221] Among the motion information merge candidates included in the motion information merge list, the motion information of the motion information merge candidate with the best cost can be set as the motion information of the current block. In addition, index information (e.g., merge index) indicating the motion information merge candidate selected among multiple motion information merge candidates can be encoded and sent to the decoder.

[0222] In the decoder, the motion information merge list may be configured in the same manner as in the encoder. And, a motion information merge candidate may be selected based on a merge index decoded from a bitstream. The motion information of the selected motion information merge candidate may be set as the motion information of the current block.

[0223] Unlike the motion vector prediction list, the motion information merge list is configured as a single list regardless of the prediction direction. In other words, the motion information merge candidates included in the motion information merge list may have only L0 motion information or L1 motion information, or may have bidirectional motion information (ie, L0 motion information and L1 motion information).

[0224] The prediction block of the current block may be generated through intra prediction or inter prediction. Information indicating whether the prediction mode of the current block is intra prediction or inter prediction may be encoded and signaled. Here, the current block may be a CU, a TU, or a PU.

[0225] At the same time, prediction information can be allocated to each sub-block within the current block based on information around the current block. Specifically, after the current block is divided into a plurality of sub-blocks, prediction information can be allocated to each sub-block in the sub-blocks.

[0226] Here, the prediction information may be different depending on the prediction mode of the current block. As an example, when intra prediction is applied to the current block, the prediction information may include the intra prediction mode. On the other hand, when inter prediction is applied to the current block, the prediction information may include at least one of a motion vector, a reference picture index, a prediction direction flag, or bidirectional weight information.

[0227] Hereinafter, an example of assigning prediction information to each of the subblocks will be described in detail.

[0228] Fig.21is a flowchart of a process of allocating prediction information to each subblock when intra prediction is applied to a current block.

[0229] Reference Fig.21 First, the intra prediction mode may be collected from the neighboring blocks of the current block (S2110). Here, the neighboring blocks may include at least one of the neighboring blocks adjacent to the current block or the neighboring sub-blocks adjacent to the current block and / or the extended boundary of the current block.

[0230] Fig. 22 is a diagram for describing an example of collecting intra prediction modes around a current block.

[0231] As in Fig. 22 In the example shown in (a), considering the actual encoding / decoding position and / or size of the neighboring blocks adjacent to the current block, an intra-frame prediction mode can be collected from each of the neighboring blocks and stored. Specifically, information about the position of the neighboring block and the intra-frame prediction mode applied to the corresponding neighboring block can be stored.

[0232] Alternatively, as in Fig. 22 In the example shown in (b), the intra prediction mode can be collected from each of the neighboring subblocks in units of blocks of a predefined size, regardless of the actual encoding / decoding position and / or size of the neighboring block. Here, the predefined size may be 4×4 or 8×8, etc. As an example, when the predefined size is 4×4, information about the intra prediction mode and the subblock position can be collected from each of the 4×4-sized subblocks around the current block and stored.

[0233] At the same time, as in Fig. 22 In the example shown in (b), the neighboring subblocks in which the intra prediction modes are collected may include not only subblocks adjacent to the current block but also subblocks adjacent to the extended boundary when the width or height of the current block is doubled.

[0234] exist Fig. 22 In the example shown in (a), there are seven neighboring blocks adjacent to the current block. Fig. 22 In (a), up to 7 pairs of block positions and intra prediction modes can be collected.

[0235] exist Fig. 22 In the example shown in (b), it is shown that the total number of sub-blocks at the top position or the left position of the current block is 13. Fig. 22 In (b), up to 13 block position and intra prediction mode pairs can be collected.

[0236] The information about the block position may include at least one of an address assigned to the corresponding block or the coordinates of a specific sample within the corresponding block. Here, the address represents a value set by sequentially numbering adjacent blocks. As an example, [x, 0] may represent an address assigned to an adjacent block adjacent to the top of the current block. Specifically, for adjacent blocks adjacent to the top of the current block, addresses may be assigned to ensure that the value of x increases in sequence from left to right. As an example, an address of [0, 0] may be assigned to the leftmost block among the top adjacent blocks, and an address of [1, 0] may be assigned to the right block of the leftmost block.

[0237] In addition, [0, y] represents the address assigned to the adjacent block adjacent to the left side of the current block. Specifically, for the adjacent block adjacent to the left side of the current block, addresses can be assigned to ensure that the value of y increases in sequence from top to bottom. As an example, an address of [0, 0] can be assigned to the topmost block among the adjacent blocks on the left side, and an address of [0, 1] can be assigned to the bottom block of the topmost block.

[0238] In addition, the specific sample may be located at the upper left corner, the upper right corner, the lower left corner, the lower right corner, or the center within the block. The coordinates of the specific sample may be calculated based on the position of the current block or the picture position. As an example, the coordinates of the specific sample may be calculated by assuming that the coordinates of the upper left sample of the current block are (0, 0).

[0239] Meanwhile, when a neighboring block is unavailable, the unavailable neighboring block may not be used to derive prediction parameters. In other words, the intra prediction mode may not be collected from the unavailable neighboring block. In this case, the neighboring block may be determined to be unavailable for at least one of a case where the neighboring block is not encoded by intra prediction, a case where the neighboring block exceeds a picture boundary, or a case where the neighboring block exceeds a coding tree unit boundary, a slice boundary, or a tile boundary.

[0240] At least one of the position, size or number of neighboring blocks where information is collected can be predefined in the encoder and the decoder. Alternatively, at least one of the position, size or number of neighboring blocks as the target of prediction information collection can be adaptively determined according to the size or shape of the current block.

[0241] Based on the intra prediction mode collected from the neighboring blocks of the current block, a prediction parameter can be derived (S2120). Specifically, if information about the position and intra prediction mode of each neighboring block in the neighboring blocks is collected, a prediction parameter that minimizes the difference between the position of the neighboring block and the intra prediction mode can be derived.

[0242] Equation 5 represents the formula for deriving the prediction parameters.

[0243] [Formula 5]

[0244] INTRA_MODE[x,y]=α1x+α2y+β

[0245] In the above formula 5, INTRA_MODE[x,y] is expressed as the intra-frame prediction mode of the neighboring block at the position of [x,y]. [x,y] represents the position of the neighboring block. According to the method of collecting position information, [x,y] can represent the address assigned to the neighboring block, or can represent the position of a specific sample within the neighboring block. In the above formula 5, a linear regression method can be used to derive prediction parameters α1, α2, and β. As an example, a linear regression method such as a least square method can be used to derive prediction parameters α1, α2, and β.

[0246] The prediction parameters include weights (α1 and α2) and offsets β. The weights may be values ​​that are multiplied by the positions, and the offsets may be values ​​that are added to / subtracted from the results of applying the weights. Each of the weights and offsets may be 0 or a non-zero integer.

[0247] Meanwhile, the weight α1 applied to the x-axis position and the weight α2 applied to the y-axis position may be set to the same value.

[0248] Once the prediction parameters are derived, the intra prediction mode of each subblock in the current block can be derived based on the derived prediction parameters (S2130). Specifically, the position (x, y) of the subblock in the current block can be substituted into the above formula 5 to derive the intra prediction mode INTRA_MODE[x, y] of the subblock. Here, the position of the subblock may include at least one of an address assigned to the subblock or a position of a specific sample in the subblock. As an example, when the address assigned to the subblock is expressed as [x, y], x represents the column to which the subblock belongs, and y represents the row to which the subblock belongs, each of which can be a natural number equal to or greater than 0 or 1. For example, the address assigned to the subblock located at the upper left corner of the current block can be set to [1, 1].

[0249] The specific sample may be a sample at the upper left corner, upper right corner, lower left corner, lower right corner, or center position of the sub-block.

[0250] At the same time, there may be a situation where the result output by equation 5 exceeds the range of intra prediction modes predefined in the encoder and decoder (for example, Figure 5 In order to solve this problem, the result output by equation 5 can be clipped (ie, a clipping function can be applied) to adjust it within the intra prediction mode range predefined in the encoder and decoder.

[0251] Alternatively, when the result output by equation 5 exceeds the predefined intra prediction mode range, the intra prediction mode of the current block can be obtained by adding or subtracting a predefined value. As an example, when assuming that Figure 4 When the predefined intra prediction mode range is 2 to 66 as in , if the result output by Equation 5 exceeds 66, the content obtained by subtracting the number of directional intra prediction modes (ie, 65) from the output result may be set as the intra prediction mode of the current block. For example, when the result output by Equation 5 has a value less than 0, the content obtained by adding the number of directional intra prediction modes (ie, 65) to the output result may be set as the intra prediction mode of the current block.

[0252] Alternatively, when the result output by Equation 5 exceeds a predefined intra prediction mode range, an intra prediction mode having a minimum difference with the output result may be set as the intra prediction mode of the subblock.

[0253] When an intra prediction mode is assigned to each subblock, intra prediction may be independently performed for each subblock.

[0254] Meanwhile, the size of the sub-block may be predefined in the encoder and the decoder. As an example, the sub-block may have a size of 4×4 or 8×8.

[0255] Alternatively, the size or shape of the sub-block may be adaptively determined according to the size or shape of the current block. As an example, when the size of the current block is less than a threshold, the sub-block may be determined to have a size of 4×4, and when the size of the current block is greater than the threshold, the sub-block may be determined to have a size of 8×8. Here, the size of the current block may represent at least one of a width, a height, or a product of a width and a height.

[0256] Fig.23 is a flowchart of a process of allocating prediction information to each subblock when inter prediction is applied to a current block.

[0257] Reference Fig.23 First, motion vectors can be collected from neighboring blocks of the current block (S2310). Here, as Fig. 22 As described, the neighboring block may include at least one of a neighboring block adjacent to the current block, the current block, and / or a neighboring sub-block adjacent to an extended boundary of the current block.

[0258] Meanwhile, considering that reference pictures of neighboring blocks may be different, when the reference pictures of the neighboring blocks are not the base reference pictures, the motion vectors of the neighboring blocks may be scaled, and then, the scaled motion vectors may be collected.

[0259] Here, the base reference picture may be a reference picture having a reference picture index of 0 among reference pictures or a reference picture having a smallest distance (ie, a picture order count (POC) difference) from the current picture.

[0260] The motion vector of the neighboring block may be scaled based on an output order (ie, POC) difference between a base reference picture and a current picture and an output order difference between the current picture and a reference picture of the neighboring block.

[0261] At the same time, together with the motion vectors of the neighboring blocks, the positions of the neighboring blocks may also be collected. Here, the information about the block positions may include at least one of an address assigned to the corresponding block or coordinates of a specific sample within the corresponding block.

[0262] Based on the motion vectors collected from the neighboring blocks of the current block, prediction parameters can be derived (S2320). Specifically, when the position and motion vector of each neighboring block are collected, prediction parameters that minimize the difference between the position and motion vector of the neighboring block can be derived.

[0263] Equation 6 represents the formula for deriving the prediction parameters.

[0264] [Formula 6]

[0265]

[0266] In the above formula 6, each of MV_X[x,y] and MV_Y[x,y] is expressed as the horizontal component and the vertical component of the motion vector of the neighboring block at the position of [x,y]. According to the method of collecting the position information, [x,y] may represent the address assigned to the neighboring block, or may represent the position of a specific sample within the neighboring block. In the above formula 6, a linear regression method may be used to derive prediction parameters α1, α2, α3, α4, β1, and β2. As an example, a linear regression method such as a least square method may be used to derive prediction parameters α1, α2, α3, α4, β1, and β2.

[0267] The prediction parameters include weights (α1, α2, α3, and α4) and offsets (β1 and β2). The prediction parameters may include a pair of weights (α1 and α3) multiplied by the x-coordinate and a pair of weights (α2 and α4) multiplied by the y-coordinate. The weights may be values ​​multiplied by the position, and the offsets may be values ​​added to / subtracted from the result of applying the weights.

[0268] When the prediction parameters are derived, the motion vector of each sub-block in the current block can be derived based on the derived prediction parameters (S2330). Specifically, the position (x, y) of the sub-block in the current block can be substituted into the above formula 6 to derive the motion vector (MV_X[x, y], MV_Y[x, y]) of the sub-block. Here, the position of the sub-block may include at least one of an address assigned to the sub-block or a position of a specific sample in the sub-block. As an example, when the address assigned to the sub-block is expressed as [x, y], x may represent the column to which the sub-block belongs, and y may represent the row to which the sub-block belongs.

[0269] The specific sample may be a sample at the upper left corner, upper right corner, lower left corner, lower right corner, or center position of the sub-block.

[0270] At the same time, in order to prevent the magnitude of the horizontal motion vector and the vertical motion vector output by equation 6 from exceeding the threshold, the output result may be clipped. Here, the threshold may be a value predefined in the encoder and the decoder, or may be adaptively determined according to the bit depth, the size of the picture, or the color component.

[0271] When a motion vector is assigned to each sub-block, inter-frame prediction can be performed independently for each sub-block. In this case, inter-frame prediction of the sub-block can be performed based on the reference picture. Specifically, the sub-reference block specified based on the motion vector of the sub-block in the reference picture can be set as the prediction block of the sub-block.

[0272] Meanwhile, the size of the sub-block may be predefined in the encoder and the decoder. As an example, the sub-block may have a size of 4×4 or 8×8.

[0273] The method based on Fig.23 The motion vector derivation method based on the prediction parameters shown in . As an example, it may be determined whether to apply the motion vector derivation method based on the prediction parameters independently for each of the L0 direction and the L1 direction.

[0274] When the motion vector derivation method based on the prediction parameter is applied to the L0 direction, the prediction parameter can be derived by collecting the motion vector in the L0 direction from the neighboring blocks of the current block. On the other hand, when the motion vector derivation method based on the prediction parameter is applied to the L1 direction, the prediction parameter can be derived by collecting the motion vector in the L1 direction from the neighboring blocks of the current block.

[0275] Alternatively, after determining whether L0 prediction is applied to the current block, whether L1 prediction is applied, or whether bidirectional prediction is applied, it may be determined whether a motion vector derivation method based on prediction parameters is applied only to a direction applied to the current block.

[0276] In order to determine whether at least one of L0 prediction or L1 prediction is applied to the current block, at least one of an L0 prediction flag indicating whether L0 prediction is applied, an L1 prediction flag indicating whether L1 prediction is applied, a bidirectional prediction flag indicating whether bidirectional prediction is applied, or a prediction direction index indicating one of the prediction direction candidates may be encoded and signaled.

[0277] Meanwhile, when collecting motion vectors from neighboring blocks, there may be a case where the neighboring blocks do not have motion information of a prediction direction to be collected.

[0278] As an example, when the prediction parameter-based motion vector derivation method is applied to the L0 direction, the L0 motion vector must be collected from the neighboring blocks of the current block. In this case, when the L0 motion vector does not exist in the neighboring block and only the L1 motion vector exists, or when the neighboring block is encoded by intra prediction, the corresponding neighboring block may not be used to derive the prediction parameter.

[0279] Alternatively, when the L0 motion vector does not exist in the neighboring block and only the L1 motion vector exists, the content obtained by scaling the L1 reference picture according to the output order of the base reference picture on the L0 reference picture list (for example, the reference picture with an index of 0 among the reference pictures or the reference picture closest to the current picture) can be used to derive the prediction parameters.

[0280] In the above examples, it is described that the prediction parameters of the current block are derived based on the information collected from the neighboring blocks, and the prediction information of each sub-block in the current block is derived based on the derived prediction parameters.

[0281] Different from the described example, the prediction parameters of the current block may also be derived based on the prediction parameters of neighboring blocks.

[0282] Specifically, the prediction parameters of the current block can be set to be the same as the prediction parameters of the neighboring blocks adjacent to the current block. The neighboring blocks may include at least one of the top neighboring block, the left neighboring block, the upper left neighboring block, the upper right neighboring block or the lower left neighboring block of the current block.

[0283] Alternatively, a prediction parameter candidate may be derived from at least one neighboring block adjacent to the current block, and a candidate list may be generated based on the prediction parameter candidate. In this case, the at least one neighboring block may include at least one of a top neighboring block, a left neighboring block, an upper left neighboring block, an upper right neighboring block, or a lower left neighboring block of the current block. After setting the prediction parameter of the neighboring block as the prediction parameter candidate, the corresponding prediction parameter candidate may be inserted into the candidate list.

[0284] When a plurality of prediction parameter candidates are inserted into the candidate list, index information indicating one of the plurality of prediction parameter candidates may be encoded and signaled. The prediction parameter candidate selected by the index information may be set as a prediction parameter of the current block.

[0285] Alternatively, the prediction parameter candidate selected by the index information can be set as the prediction value of the prediction parameter of the current block, and the difference value of the prediction parameter of the block can be added to the prediction value to derive the prediction parameter of the current block. For this purpose, the difference between the prediction parameter candidate and the prediction parameter of the current block can be encoded and signaled.

[0286] Information showing whether the prediction parameters of the current block are actively derived from the prediction information collected from the neighboring blocks or whether the prediction parameters of the current block are derived based on the prediction parameters of the neighboring blocks can be encoded and signaled. As an example, this information can be a 1-bit flag.

[0287] Based on at least one of the size or shape of the current block, it may be determined whether it is allowed to derive the prediction parameters of the current block by prediction information collected from neighboring blocks, or whether it is allowed to derive the prediction parameters of the current block based on the prediction parameters of the neighboring blocks. As an example, when it is not allowed for the current block to derive the prediction parameters of the current block by prediction information collected from neighboring blocks, decoding / encoding of the flag may be omitted, and it may be determined to derive the prediction parameters of the current block based on the prediction parameters of the neighboring blocks. Alternatively, when it is not allowed for the current block to derive the prediction parameters of the current block based on the prediction parameters of the neighboring blocks, encoding / decoding of the flag may be omitted, and it may be determined to derive the prediction parameters of the current block based on the prediction information collected from the neighboring blocks.

[0288] Alternatively, since the process of deriving prediction parameters is omitted, prediction information (ie, intra prediction mode or motion vector) of each subblock in a neighboring block adjacent to the current block may be set as prediction information of each subblock in the current block.

[0289] exist Fig.21 and Fig.23 In the description, it is described that prediction information is derived for each sub-block in the current block based on prediction parameters. Fig.21 and Fig.23 In the implementation method of , instead of deriving prediction information in units of sub-blocks, prediction information may be derived in units of blocks. In other words, when deriving prediction parameters for the current block, the position of a specific sample in the current block may be inserted into Equation 5 or Equation 6 to derive prediction information for the current block. Here, the specific sample may be a sample at the upper left corner, upper right corner, lower left corner, lower right corner, or center position of the current block. Deriving prediction information for the current block instead of sub-block units is the same as assigning the same prediction information to all sub-blocks within the current block.

[0290] Information showing a unit in which prediction information is derived (ie, whether prediction information is derived in units of blocks or in units of sub-blocks) may be encoded and signaled.

[0291] Alternatively, the unit for deriving the prediction information may be adaptively determined based on at least one of the color component to be encoded / decoded, the ratio of the width to the height of the current block, or the size of the current block. As an example, when encoding / decoding a luma block, the prediction information may be derived in units of sub-blocks. On the other hand, when encoding / decoding a chroma block, the prediction information may be derived in units of blocks. In other words, when encoding / decoding a chroma block, one prediction information may be assigned to the entire chroma block by using the position of a specific sample in the chroma block.

[0292] When the motion of an object occurs, such as size change or rotation, an affine model can be used to compensate for the motion of the object. When the affine model is applied to the current block, the control point motion vector of the affine model can also be derived based on the prediction parameters of the current block. At the same time, information showing whether the affine model is applied to the current block can be encoded and signaled. The information can be an index or a 1-bit flag indicating one of multiple motion candidates.

[0293] The affine model using 6 parameters can be expressed as the following Equation 7.

[0294] [Formula 7]

[0295]

[0296] In the above formula 7, (mv 0x , mv 0y )、(mv 1x , mv 1y ) and (mv 2x , mv 2y ) represents the control point motion vector (CPMV) of the current block. (mv 0x , mv 0y )、(mv 1x , mv 1y ) and (mv 2x , mv 2y ) can be represented as v0, v1 and v2. In addition, in Formula 7, W and H represent the width and height of the current block, respectively.

[0297] The affine model using 4 parameters can be expressed as the following Equation 8.

[0298] [Formula 8]

[0299]

[0300] Based on the control point motion vector of the current block, the motion vector (MV_X[x,y], MV_Y[x,y]) of the sub-block included in the sample at the position [x,y] or at the position [x,y] in the current block can be derived.

[0301] Fig.24 The control point motion vectors of the current block are shown.

[0302] When applying a 6-parameter affine model, such as Fig.24 In the example shown in (a), three control point motion vectors can be used. Specifically, v0 (the motion vector of the control point corresponding to the upper left corner of the current block), v1 (the motion vector of the control point corresponding to the upper right corner), and v2 (the motion vector of the control point corresponding to the lower left corner) can be used.

[0303] At the same time, when applying a 4-parameter affine model, you can use Fig.24 (b) shows two control point motion vectors. Specifically, v0 (the motion vector of the control point corresponding to the upper left corner of the current block) and v1 (the motion vector of the control point corresponding to the upper right corner) can be used. Alternatively, unlike the example shown, v0 (the motion vector of the control point corresponding to the upper left corner of the current block) and v2 (the motion vector of the control point corresponding to the lower left corner) can also be used.

[0304] The control point motion vector of the current block can be derived as a combination of translational motion vectors of one or more neighboring blocks encoded by the affine model.

[0305] As another example, the control point motion vector of the current block can be derived by using the prediction parameters of the current block. Specifically, based on Equation 6, prediction parameters α1, α2, α3, α4, β1, and β2 derived based on the motion vectors of neighboring blocks adjacent to the current block can be derived. Then, according to the number of parameters of the affine model, each of the derived prediction parameters α1, α2, α3, α4, β1, and β2 can be matched with each item in Equation 7 or Equation 8. As an example, when a 6-parameter affine model is applied, each prediction parameter can be matched with each item in Equation 7 to set the correlation between the prediction parameter and the control point motion vector, as shown in Equation 9 below.

[0306] [Formula 9]

[0307]

[0308] β1=mv 0x

[0309] β2=mv 0y

[0310] Based on the above formula 9, we can get the first control point motion vector v0 of the current block (i.e. (mv 0x , mv 0y )) and the second control point motion vector v1 (ie (mv 1x , mv 1y)) and the third control point motion vector v2 (ie (mv 2x , mv 2y )).

[0311] Alternatively, when a 4-parameter affine model is applied, the number of prediction parameters may be adjusted to match the number of variables of the 4-parameter affine model. Specifically, Equation 6 may be modified as in Equation 10 below.

[0312] [Formula 10]

[0313]

[0314] Each prediction parameter can then be matched with each term of Equation 8 to set the correlation between the prediction parameter and the control point motion vector, as shown in Equation 11 below.

[0315] [Formula 11]

[0316]

[0317] β1=mv 0x

[0318] β2=mv 0y

[0319] Based on the above formula 11, we can get the first control point motion vector v0 of the current block (i.e. (mv 0x , mv 0y )) and the second control point motion vector v1 (ie (mv 1x , mv 1y )).

[0320] Each control point motion vector derived based on the prediction parameters may also be set as a predicted value of the control point motion vector of the current block. In this case, the control point motion vector of the current block may be derived by adding the difference of the control point motion vectors to the control point motion vector derived based on the prediction parameters. In addition, the encoder may encode and signal information showing the difference of the control point motion vectors.

[0321] The grammatical terms used in the above embodiments are only named for the convenience of description.

[0322] When an embodiment described based on a decoding process or an encoding process is applied to an encoding process or a decoding process, it is included in the scope of the present disclosure. When an embodiment described in a predetermined order is changed in an order different from the description, it is also included in the scope of the present disclosure.

[0323] The above-mentioned embodiments are described based on a series of steps or flow charts, but they do not limit the time series order of the present disclosure, and if necessary, they can be performed simultaneously or in different orders. In addition, each component (e.g., unit, module, etc.) of the block diagram in the above-mentioned embodiments can be implemented as a hardware device or software, and a plurality of components can be combined and implemented as a hardware device or software. The above-mentioned embodiments can be implemented into the form of program instructions that can be executed by various computer components, and recorded in a computer-readable recording medium. The computer-readable recording medium can include program instructions, data files, data structures, etc. individually or in combination. Magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs, DVDs, magneto-optical media such as optical floppy disks, and hardware devices such as ROMs, RAMs, flash memories, etc. that are specially configured to store and execute program instructions are included in computer-readable recording media. The hardware device can be configured to operate as one or more software modules to perform processing according to the present disclosure, and the software device can be configured to operate as one or more hardware modules to perform processing according to the present disclosure.

[0324] Industrial Applicability

[0325] The embodiments of the present disclosure may be applied to electronic devices that encode or decode images.

Claims

1. A method for decoding an image, the method comprising: Collect prediction information from neighboring blocks of the current block; Based on the collected prediction information, deriving prediction parameters of the current block; as well as Based on the prediction parameters, prediction information of each sub-block in the current block is obtained.

2. The method according to claim 1, wherein: When the prediction mode of the current block is intra prediction, the prediction information is the intra prediction mode.

3. The method according to claim 2, wherein: The prediction parameters include a pair of weights and biases, and The intra prediction mode of the subblock is obtained by summing a first value obtained by multiplying the x-axis position of the subblock and a first weight, a second value obtained by multiplying the y-axis position of the subblock and a second weight, and an offset.

4. The method according to claim 3, wherein: When a result value obtained by summing the first value, the second value, and the offset exceeds a range of a predefined intra prediction mode, the result value is clipped to derive the intra prediction mode of the subblock.

5. The method according to claim 3, wherein: When a result value obtained by summing the first value, the second value and the offset exceeds a range of a predefined intra prediction mode, the intra prediction mode of the subblock is derived by adding a predefined constant to the result value or subtracting the predefined constant from the result value.

6. The method according to claim 1, wherein: When the prediction mode of the current block is inter-frame prediction, the prediction information is a motion vector.

7. The method according to claim 6, wherein: When the reference picture of the neighboring blocks is different from the base reference picture, the scaled motion vectors of the neighboring blocks are collected.

8. The method according to claim 7, wherein: The base reference picture is the first reference picture in the reference picture list or a reference picture in the reference picture list that is closest to the current picture.

9. The method according to claim 6, wherein: The prediction parameters are derived independently for each of the L0 direction and the L1 direction.

10. The method according to claim 6, wherein: When applying inter-frame prediction based on an affine model to the current block, deriving a control point motion vector of the current block based on the prediction parameters, and Wherein, a motion vector of each sub-block in the sub-blocks is obtained based on the control point motion vector.

11. The method according to claim 1, wherein: The image decoding method further includes: determining whether to derive the prediction parameter based on the collected prediction information, and When it is determined not to derive the prediction parameter based on the collected prediction information, the prediction parameter is derived from a neighboring block adjacent to the current block.

12. The method according to claim 11, wherein: The prediction parameters of the current block are set to be the same as the prediction parameters of the neighboring blocks.

13. The method according to claim 11, wherein: A neighboring block used by the current block to derive a prediction parameter is specified based on the index information.

14. A method for encoding an image, the method comprising: Collect prediction information from neighboring blocks of the current block; Based on the collected prediction information, deriving prediction parameters of the current block; as well as Based on the prediction parameters, prediction information of each sub-block in the current block is obtained.

15. A computer-readable recording medium, the computer-readable recording medium recording a bit stream generated by an image encoding method, the image encoding method comprising: Collect prediction information from neighboring blocks of the current block; Based on the collected prediction information, deriving prediction parameters of the current block; as well as Based on the prediction parameters, prediction information of each sub-block in the current block is obtained.

Images