Image encoding / decoding method and apparatus using intra prediction

Abstract

The video signal processing method and apparatus according to the present invention can determine an intra prediction mode of a current block, determine a reference sample for intra prediction of the current block, determine a predetermined matrix based on the intra prediction mode, and predict the current block based on the reference sample and the matrix.

Description

[0001] Description of the case

[0002] This application is a divisional application based on the patent with application date of September 6, 2019, application number 201980056309.3, and invention name “Image encoding / decoding method and device using intra-frame prediction”. Technical Field

[0003] The present invention relates to an image encoding / decoding method and device. Background Art

[0004] With the widespread use of the Internet and portable devices, as well as the advancement of information and communication technologies, the use of multimedia data is rapidly increasing. Consequently, the demand for improved performance and efficiency of image processing systems is increasing significantly, as they enable the execution of various services and tasks through image prediction in various systems. However, research and development efforts to address this need are insufficient.

[0005] Thus, in the conventional image encoding / decoding methods and apparatuses, there is a need to improve image processing, and in particular, to improve the performance of image encoding or image decoding. Summary of the Invention

[0006] Technical issues

[0007] The present invention aims to improve encoding / decoding efficiency through adaptive block partitioning.

[0008] The present invention aims to improve encoding / decoding efficiency through matrix-based intra-frame prediction.

[0009] The present invention provides a method and apparatus for determining reference samples and matrices for matrix-based intra prediction.

[0010] The present invention provides a method and apparatus for downsampling and upsampling for matrix-based intra-frame prediction.

[0011] Technical Solution

[0012] According to the video signal processing method and device of the present invention, the intra-frame prediction mode of the current block can be determined, the reference sample used for the intra-frame prediction of the current block can be determined, a predetermined matrix can be determined based on the intra-frame prediction mode, and the current block can be predicted based on the reference sample and the matrix.

[0013] In the video signal processing method and apparatus according to the present invention, determining the reference sample may include: determining an adjacent area of ​​the current block and down-sampling the determined adjacent area.

[0014] In the video signal processing method and apparatus according to the present invention, the adjacent areas are divided into a plurality of sample groups, each of which is composed of one or more samples. A representative value of the sample group is determined as the reference sample, and the representative value can be any one of an average value, a minimum value, a maximum value, a mode, or a median value.

[0015] In the video signal processing method and apparatus according to the present invention, the matrix is ​​determined by further considering encoding information of the current block, where the encoding information may include size, shape, angle or directionality of intra-prediction mode of the current block.

[0016] In the video signal processing method and apparatus according to the present invention, predicting the current block may include: generating a predicted block by applying the matrix to the reference sample.

[0017] In the video signal processing method and apparatus according to the present invention, predicting the current block may further include rearranging all or part of the prediction samples of the generated prediction block.

[0018] In the video signal processing method and apparatus according to the present invention, predicting the current block may further include: interpolating the current block based on the prediction block or at least one of reconstructed samples adjacent to the current block.

[0019] Technical Effects

[0020] According to the present invention, encoding / decoding efficiency can be improved by segmenting tree-structured blocks.

[0021] According to the present invention, encoding / decoding efficiency can be improved through matrix-based intra prediction.

[0022] According to the present invention, encoding / decoding efficiency can be improved through downsampling or upsampling for matrix-based intra prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic block diagram of an encoding device as an embodiment of the present invention is shown.

[0024] Figure 2 A schematic block diagram of a decoding device as an embodiment of the present invention is shown.

[0025] Figure 3 A block partitioning type is shown as an embodiment to which the present invention is applicable.

[0026] Figure 4 A block partitioning method based on a tree structure is shown as an embodiment to which the present invention is applied.

[0027] Figure 5 The process of performing intra-frame prediction on the current block based on a matrix as an embodiment of the present invention is shown.

[0028] Figure 6 A method for determining reference samples by downsampling adjacent areas is shown as an embodiment applicable to the present invention.

[0029] Figure 7 A downsampling method based on weighted average is shown as an embodiment to which the present invention is applicable.

[0030] Figure 8 The diagram shows a method for dividing the first prediction sample and interpolating the remaining areas as an embodiment applicable to the present invention.

[0031] Figure 9 As one embodiment to which the present invention is applicable, a method of assigning a weight value to a distance in an interpolation step is shown.

[0032] Figure 10 The sequence of interpolation steps is shown as one embodiment to which the present invention is applicable. DETAILED DESCRIPTION

[0033] According to the video signal processing method and device of the present invention, the intra-frame prediction mode of the current block can be determined, the reference sample used for the intra-frame prediction of the current block can be determined, a predetermined matrix can be determined based on the intra-frame prediction mode, and the current block can be predicted based on the reference sample and the matrix.

[0034] In the video signal processing method and apparatus according to the present invention, determining the reference sample may include: determining an adjacent area of ​​the current block; and down-sampling the determined adjacent area.

[0035] In the video signal processing method and apparatus according to the present invention, the adjacent areas are divided into a plurality of sample groups, each of which is composed of one or more samples. A representative value of the sample group is determined as the reference sample, and the representative value can be any one of an average value, a minimum value, a maximum value, a mode, or a median value.

[0036] In the video signal processing method and apparatus according to the present invention, the matrix is ​​determined by further considering encoding information of the current block, where the encoding information may include size, shape, angle or directionality of intra-prediction mode of the current block.

[0037] In the video signal processing method and apparatus according to the present invention, predicting the current block may include: generating a predicted block by applying the matrix to the reference sample.

[0038] In the video signal processing method and apparatus according to the present invention, predicting the current block may further include rearranging all or part of the prediction samples of the generated prediction block.

[0039] In the video signal processing method and apparatus according to the present invention, predicting the current block may further include: interpolating the current block based on the prediction block or at least one of reconstructed samples adjacent to the current block.

[0040] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the accompanying drawings and described in detail. However, it should be understood that these specific embodiments are not intended to limit the present invention to specific implementations and encompass all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. Similar reference numerals are used to identify similar components in each of the drawings.

[0041] Terms such as first and second may be used to describe constituent elements, but the constituent elements should not be limited by the terms. The terms are only used to distinguish one constituent element from other constituent elements. For example, the first constituent element may be named the second constituent element without departing from the scope of the present invention, and similarly, the second constituent element may be named the first constituent element. The so-called and / or term refers to a combination of multiple related items or any one of the multiple related items recorded.

[0042] When a component is described as being “connected” or “linked” to another component, it should be understood that it may be directly connected or linked to the other component, or other components may be present between the component and the other component. Conversely, when a component is described as being “directly connected” or “directly linked” to another component, it should be understood that no other components exist between the component and the other component.

[0043] The terms used in this application are only used to illustrate specific embodiments and are not intended to limit the present invention. Unless otherwise expressly indicated in the text, the singular expression includes the plural expression. In this application, the terms "comprising" or "having" should be understood to indicate the presence of the features, numbers, steps, actions, constituent elements, parts or their combinations described in the specification, and do not preclude the presence or additional possibility of one or more other features, numbers, steps, actions, constituent elements, parts or their combinations.

[0044] Unless otherwise defined, all terms, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries should be interpreted as having the same meaning as in the relevant technical literature, and if not explicitly defined in this application, their meanings should not be interpreted as ideal or overly formal.

[0045] Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, the same reference numerals will be used for the same components in the accompanying drawings, and repeated descriptions of the same components will be omitted.

[0046] Figure 1 A schematic block diagram of an encoding device as an embodiment of the present invention is shown.

[0047] refer to Figure 1 The encoding device 100 may include: an image segmentation unit 110, prediction units 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filtering unit 150 and a memory 155.

[0048] Figure 1 Each component shown in the figure is shown separately to represent different characteristic functions in the image encoding device, and this may mean that each component is composed of separate hardware. However, for ease of description, each component is listed as a component and included. At least two components in each component are combined into one component, or one component can be divided into multiple components to perform functions. Such comprehensive embodiments of each component and individual embodiments are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

[0049] In addition, some components may be optional components that are merely used to improve performance, and are not essential components for performing the basic functions of the present invention. The present invention may be implemented by only including components that are essential to the implementation of the present invention, excluding components that are merely used to improve performance. Structures that only include essential components excluding optional components that are merely used to improve performance are also included in the scope of the present invention.

[0050] The image segmentation unit 110 can segment the input image into at least one block. In this case, the block can represent a coding unit (CU), a prediction unit (PU), or a transform unit (TU). The segmentation can be performed based on at least one of a quad tree, a binary tree, and a ternary tree. A quad tree is a method of segmenting an upper layer block into four lower layer blocks having half the width and height of the upper layer block. A binary tree is a method of segmenting an upper layer block into two lower layer blocks having at least one width or height half the width of the upper layer block. In the binary tree segmentation, by segmenting based on the aforementioned binary tree with half the height of the upper layer block, the block can have not only a square shape but also a non-square shape.

[0051] In the following embodiments of the present invention, the encoding unit may be used as a unit for performing encoding, or may be used as a unit for performing decoding.

[0052] The prediction units 120 and 125 may include an inter-frame prediction unit 120 for performing inter-frame prediction and an intra-frame prediction unit 125 for performing intra-frame prediction. Whether inter-frame prediction or intra-frame prediction is used is determined for each prediction unit, and specific information based on each prediction method (e.g., intra-frame prediction mode, motion vector, reference image, etc.) may be determined. In this case, the processing unit used to perform the prediction may be different from the processing unit used to determine the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, or the prediction may be performed by the transform unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. Furthermore, in the entropy coding unit 165, prediction mode information, motion vector information, etc. used for prediction may be encoded along with the residual value and transmitted to the decoding device. When a certain coding mode is used, the original block may be encoded as is and transmitted to the decoding unit without generating a prediction block through the prediction units 120 and 125.

[0053] The inter-frame prediction unit 120 can predict the prediction unit based on the information of at least one image in the previous image or the next image of the current image. Depending on the situation, the prediction unit can also be predicted based on the information of some encoded areas within the completed current image. The prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0054] The reference image interpolation unit uses reference image information provided by memory 155 to generate pixel information smaller than an integer pixel in the reference image. For luma pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information smaller than an integer pixel in 1 / 4 pixel units. For chrominance signals, a 4-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information smaller than an integer pixel in 1 / 8 pixel units.

[0055] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used as methods for calculating motion vectors. Based on the interpolated pixels, the motion vector can have a motion vector value of 1 / 2 or 1 / 4 pixel units. The motion prediction unit can predict the current prediction unit through different motion prediction methods. Various methods such as the Skip method, the Merge method, and the Advanced Motion Vector Prediction (AMVP) method can be used as motion prediction methods.

[0056] The intra-frame prediction unit 125 can generate a prediction unit based on the reference pixel information adjacent to the current block, where the reference pixel information adjacent to the current block is pixel information in the current image. Since the adjacent block of the current prediction unit is an inter-frame predicted block, when the reference pixel is an inter-frame predicted pixel, the reference pixel information included in the inter-frame predicted block can be replaced with the reference pixel information of the adjacent intra-frame predicted block. In other words, when a reference pixel is unavailable, the unavailable reference pixel information can be replaced with at least one reference pixel from the available reference pixels.

[0057] The prediction mode in intra-frame prediction can include a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode used to predict the luma component and the mode used to predict the chroma component can be different, and the chroma component can be predicted using the intra-frame prediction mode used to predict the luma component or using the predicted / reconstructed luma component.

[0058] The intra prediction method may generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to reference pixels according to an intra prediction mode. The types of AIS filters applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction modes of the adjacent prediction units located to the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the adjacent prediction units, if the intra prediction modes of the current prediction unit and the adjacent prediction unit are the same, predetermined flag information may be used to transmit information indicating that the intra prediction modes of the current prediction unit and the adjacent prediction unit are the same. If the intra prediction modes of the current prediction unit and the adjacent prediction unit are different, entropy coding may be performed to encode the intra prediction mode information of the current block.

[0059] In addition, the prediction units 120 and 125 may generate residual information including a difference between the generated prediction unit and the original block. The generated residual block may be input to the transformation unit 130.

[0060] The transform unit 130 may transform the residual block including the residual data using a transform of a type such as DCT or DST. At this time, the transform type may be determined based on the intra prediction mode of the prediction unit used to generate the residual block.

[0061] The quantization unit 135 may quantize the value transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary depending on the block or the importance of the image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0062] The rearrangement unit 160 may rearrange the coefficient values ​​of the quantized residual block. The rearrangement unit 160 may convert the coefficients of the two-dimensional block shape into a one-dimensional vector shape using a coefficient scanning method. For example, the rearrangement unit 160 may use a predetermined scanning method to scan from the DC coefficient to the coefficients of the high frequency region to convert them into a one-dimensional vector shape.

[0063] The entropy coding unit 165 may perform entropy coding based on the value calculated by the rearrangement unit 160. Entropy coding may use various coding methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0064] The entropy coding unit 165 can encode various information from the rearrangement unit 160 and the prediction units 120 and 125, such as residual coefficient information and block type information of the coding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference image information, block interpolation information, filtering information, etc.

[0065] The entropy encoding unit 165 may perform entropy encoding on the coefficient values ​​of the coding units input from the arrangement unit 160 .

[0066] The inverse quantization unit 140 and the inverse transformation unit 145 inversely quantize the values ​​quantized by the quantization unit 135 and inversely transform the values ​​transformed by the transformation unit 130. The residual values ​​generated by the inverse quantization unit 140 and the inverse transformation unit 145 can be combined with the predicted prediction unit through the motion estimation unit, motion compensation unit, and intra-frame prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0067] The filtering unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0068] A deblocking filter removes block distortion caused by boundaries between blocks in a reconstructed image. Whether to apply a deblocking filter to a block is determined based on the pixels in several columns or rows of the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be used, depending on the desired deblocking filter strength. Furthermore, when applying a deblocking filter, both vertical and horizontal filtering can be performed in parallel.

[0069] The offset correction unit can correct the offset of the deblocked image relative to the original image on a pixel-by-pixel basis. To perform offset correction on a specific image, a method can be used in which pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or a method can be used in which the offset is applied by considering edge information of each pixel.

[0070] ALF (Adaptive Loop Filtering) can be performed based on the comparison value of the filtered reconstructed image and the original image. After the pixels included in the image are divided into predetermined groups, a filter applicable to the group is determined, and filtering can be performed differently for each group. For information on whether ALF is applicable, each coding unit (Coding Unit, CU) can transmit a luminance signal, and the shape of the ALF filter to be applied and the filter coefficients can be changed for each block. In addition, regardless of the characteristics of the applicable object block, an ALF filter of the same shape (fixed shape) can also be applied.

[0071] The memory 155 may store the reconstructed block or image calculated by the filtering unit 150 , and the stored reconstructed block or image may be provided to the prediction units 120 and 125 when performing inter-frame prediction.

[0072] Figure 2 A schematic block diagram of a decoding device as an embodiment of the present invention is shown.

[0073] refer to Figure 2 The decoding device 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transformation unit 225, prediction units 230 and 235, a filtering unit 240, and a memory 245.

[0074] Figure 2 Each component shown in the figure is shown separately to represent different characteristic functions in the decoding device, and this may mean that each component is composed of separate hardware. However, for ease of description, each component is listed as a component and included. At least two components in each component are combined into one component, or one component can be divided into multiple components to perform functions. Such comprehensive embodiments of each component and individual embodiments are also included in the scope of the present invention as long as they do not depart from the essence of the present invention.

[0075] The entropy decoding unit 210 may perform entropy decoding on the input bitstream. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) may be applied for entropy decoding.

[0076] The entropy decoding unit 210 may decode information regarding intra-frame prediction and inter-frame prediction performed by the encoding apparatus.

[0077] The rearrangement unit 215 can rearrange the code stream entropy-decoded by the entropy decoding unit 210. The rearrangement can be performed by reconstructing coefficients represented in the form of a one-dimensional vector into coefficients in the form of a two-dimensional block. The rearrangement unit 215 can receive information about coefficient scanning performed by the encoding device and perform rearrangement by performing a reverse scan based on the scanning order performed by the encoding device.

[0078] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter and the rearranged coefficient values ​​of the block.

[0079] The inverse transform unit 225 may perform inverse transform on the inverse quantized transform coefficient using a predetermined transform type. The transform type may be determined based on at least one of information on a prediction mode (inter / intra prediction), block size / shape, intra prediction mode, component type (luminance / color difference component), and partition type (QT, BT, TT, etc.).

[0080] The prediction units 230 and 235 may generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit 210 and previously decoded block or image information provided by the memory 245 .

[0081] The prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information, such as prediction unit information input by the entropy decoding unit 210, information related to the intra-frame prediction mode of the intra-frame prediction method, information related to the motion prediction of the inter-frame prediction method, etc., to distinguish prediction units in the current coding unit (CU) and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 may utilize the information required for inter-frame prediction of the current prediction unit provided by the encoding device, and perform inter-frame prediction for the current prediction unit based on information included in at least one of the previous image or the next image of the current image including the current prediction unit. Alternatively, inter-frame prediction may be performed based on information of some reconstructed areas in the current image including the current prediction unit. To this end, the reconstructed areas may be added to a reference image list.

[0082] In order to perform inter-frame prediction, based on the coding unit, it can be determined whether the motion prediction method of the prediction unit included in the coding unit is the skip mode (Skip Mode), merge mode (Merge Mode), AMVP mode (AMVP Mode), or current image reference mode.

[0083] The intra prediction unit 235 can generate a prediction block based on the pixel information in the current image. When the prediction unit is a prediction unit that performs intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit provided by the encoding device. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a component that filters the reference pixels of the current block. It can determine whether to apply the filter based on the prediction mode of the current prediction unit. The prediction mode of the prediction unit provided by the encoding device and the AIS filter information can be used to perform AIS filtering on the reference pixels of the current block. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0084] When the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on the value of the pixel interpolated from the reference pixel, the reference pixel interpolation unit may interpolate the reference pixels to generate reference pixels in pixel units with a value less than an integer. When the prediction mode of the current prediction unit is a prediction mode that does not interpolate the reference pixels to generate a prediction block, the reference pixels may not be interpolated. When the prediction mode of the current block is the DC mode, the DC filter may generate a prediction block through filtering.

[0085] The reconstructed block or image may be provided to the filtering unit 240. The filtering unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0086] The encoding device may receive information about whether a deblocking filter is applied to the block or image, and, if a deblocking filter is applied, information about whether a stronger filter or a weaker filter is applied. The deblocking filter of the decoding device may receive the deblocking filter-related information provided by the encoding device, and the decoding device may perform deblocking filtering on the block.

[0087] During encoding, the offset correction unit may perform offset correction on the reconstructed image based on the offset correction type and offset value information applicable to the image.

[0088] ALF can be applied to a coding unit based on ALF applicability information provided by the encoder, ALF coefficient information, etc. Such ALF information can be provided by including it in a specific parameter set.

[0089] The memory 245 may store the reconstructed image or block as a reference image or reference block, and may provide the reconstructed image to an output portion.

[0090] Figure 3 A block partitioning type is shown as an embodiment to which the present invention is applicable.

[0091] refer to Figure 3 , blocks from a to s can be obtained according to the segmentation settings and segmentation methods, and additional block shapes not shown can also be obtained.

[0092] As an example (1), asymmetric partitioning can be allowed for tree-based partitioning. For example, for a binary tree, it can be blocks such as b and c, or it can also be blocks such as b to g. When the flag allowing asymmetric partitioning is not explicitly or implicitly activated according to the encoding / decoding settings, the available candidate blocks can be b or c. When the flag allowing asymmetric partitioning is activated, the available candidate blocks can be b, d, e (horizontal partitioning in this example), or c, f, g (vertical partitioning in this example).

[0093] In the example, it is assumed that the length ratio of the asymmetrically divided left:right or top:bottom is 1:3 or 3:1, but it is not limited to this. There may also be candidate groups with other ratios (for example, 1:2, 1:4, 2:3, 2:5, 3:5, etc.) depending on the encoding settings.

[0094] Various examples of segmentation information generated in binary tree segmentation (candidate groups of 1:1, 1:3, and 3:1 in this example) are shown below.

[0095] For example, in addition to a flag indicating whether to split and a flag indicating the split direction, a flag indicating the split type can also be generated. In this case, the split type can indicate symmetric or asymmetric split. When asymmetric split is determined as the split type, a flag indicating the split ratio can be generated, and an index can be assigned based on a preset candidate group. If a split ratio of 1:3 or 3:1 is supported as a candidate group, the split ratio can be selected using a 1-bit flag.

[0096] Alternatively, in addition to the flag indicating whether to split and the flag indicating the split direction, a flag indicating the split ratio may also be generated. In this example, the candidate group for the split ratio may include candidates with a 1:1 symmetrical ratio.

[0097] In the present invention, it is assumed (when the flag allowing asymmetric partitioning is activated) that the binary tree partitioning has a structure as in the previous example. Unless otherwise specified, the binary tree represents a symmetric binary tree.

[0098] As an example (2), for tree-based segmentation, additional tree segmentation can be allowed. For example, a ternary tree, a quad type tree, an octa tree, etc. can be segmented, and n segmentation blocks can be obtained from them (3, 4, 8, and n in this example are integers). For a ternary tree, the supported blocks (when segmented into multiple blocks in this example) can be h to m, for a quad type tree, the supported blocks can be n to p, and for an octa tree, the supported blocks can be q. Whether the tree-based segmentation is supported can be implicitly determined according to the encoding / decoding settings or the relevant information can be explicitly generated. In addition, it can be used alone or mixed with binary tree, quad tree segmentation, etc. according to the encoding / decoding settings.

[0099] For example, for a binary tree, blocks such as b and c can be used. When a binary tree and a ternary tree are used in combination (in this example, it is assumed that the use range of the binary tree partially overlaps with the use range of the ternary tree), blocks such as b, c, i, and l can also be used. When the flag that allows additional splits other than the current tree is explicitly or implicitly deactivated according to the encoding / decoding settings, the available candidate blocks may be b or c. When activated, the available candidate blocks may be b, i or b, h, i, j (horizontal split in this example), or c, l or c, k, l, m (vertical split in this example).

[0100] In the example, although the description assumes that the length ratio of left: middle: right or top: middle: bottom of the ternary tree segmentation is 2:1:1 or 1:2:1 or 1:1:2, it is not limited to this and other ratios can also be set according to the encoding.

[0101] The following shows an example of segmentation information generated in ternary tree segmentation (a candidate of 1:2:1 in this example).

[0102] For example, in addition to the flag indicating whether to split and the flag indicating the split direction, a flag indicating the split type can also be generated. In this case, the split type can represent binary tree or ternary tree split.

[0103] In the present invention, adaptive encoding / decoding settings can be applied according to the segmentation method.

[0104] As an example, the partitioning method can be determined according to the type of block. For example, the coding block and the transform block can use the quadtree partitioning method, and the prediction block can use the quadtree and binary tree (or ternary tree, etc.) partitioning method.

[0105] As an example, the partitioning method can be determined based on the block size. For example, quadtree partitioning can be used for some ranges between the maximum and minimum values ​​of the block (e.g., a×b to c×d, when the latter is larger), while binary tree (or ternary tree, etc.) partitioning can be used for some ranges (e.g., e×f to g×h). In this case, range information based on the partitioning method can be explicitly generated or implicitly determined, and can also be used when the ranges overlap.

[0106] As an example, the segmentation method can be determined based on the shape of the block (or the block before segmentation). For example, if the block shape is square, quadtree and binary tree (or ternary tree, etc.) segmentation can be performed. Alternatively, if the block shape is rectangular, binary tree (or ternary tree, etc.) segmentation can be performed.

[0107] As an example, the partitioning setting can be determined based on the block type. For example, in tree-based partitioning, coding blocks and prediction blocks can use quadtree partitioning, while transform blocks can use binary tree partitioning. Alternatively, the allowed partition depth for coding blocks can be set to m, the allowed partition depth for prediction blocks can be set to n, and the allowed partition depth for transform blocks can be set to o, where m, n, and o can be the same or different.

[0108] As an example, the segmentation settings can be determined based on the size of the block. For example, quadtree segmentation can be used in some ranges of the block (e.g., a×b to c×d), binary tree segmentation can be used in some ranges (e.g., e×f to g×h; in this example, it is assumed that c×d is greater than g×h), and ternary tree segmentation can be used in some ranges (e.g., i×j to k×l; in this example, it is assumed that g×h is greater than or equal to k×l). In this case, the ranges can include all ranges between the maximum and minimum values ​​of the block, and the ranges can have non-overlapping settings or overlapping settings. For example, the minimum value of some ranges may be the same as the maximum value of some ranges, or the minimum value of some ranges may be less than the maximum value of some ranges. If there are overlapping ranges, the segmentation method with the larger maximum value can be prioritized, or information on which segmentation method to use can be explicitly generated. In other words, among the segmentation methods with a priority, whether to use a segmentation method with a lower priority can be determined based on the segmentation results, or which segmentation method to use can be determined based on the segmentation method selection information.

[0109] As an example, the segmentation settings can be determined based on the shape of the block. For example, if the block shape is square, quadtree segmentation can be used. Alternatively, if the block shape is rectangular, binary tree or ternary tree segmentation can be used.

[0110] As an example, the segmentation setting can be determined based on encoding / decoding information (e.g., slice type, color component, encoding mode, etc.). For example, when the slice type is I, quadtree (or binary tree, ternary tree) segmentation can be used within some ranges (e.g., a×b to c×d), when the slice type is P, it can be used within some ranges (e.g., e×f to g×h), and when the slice type is B, it can be used within some ranges (e.g., i×j to k×l). In addition, when the slice type is I, the allowed segmentation depth of quadtree (or binary tree, ternary tree segmentation) segmentation can be set to m, when the slice type is P, the allowed segmentation depth can be set to n, and when the slice type is B, the allowed segmentation depth can be set to o, and m, n, and o can be the same or different. Some slice types can have the same settings as other slices (e.g., P and B slices).

[0111] As another example, when the color component is a luminance component, the quadtree (or binary tree, ternary tree) segmentation allowed depth can be set to m, and when the color component is a color difference component, it can be set to n, and m and n can be the same or different. In addition, the range of the quadtree (or binary tree, ternary tree) segmentation when the color component is a luminance component (for example, a×b to c×d) and the range of the quadtree (or binary tree, ternary tree) segmentation when the color component is a color difference component (for example, e×f to g×h) can be the same or different.

[0112] As another example, when the coding mode is Intra, the allowed depth of the quadtree (or binary tree, ternary tree) split can be m, and when the coding mode is Inter, it can be n (assuming n is greater than m in this example), and m and n can be the same or different. In addition, the range of the quadtree (or binary tree, ternary tree) split when the coding mode is Intra can be the same as or different from the range of the quadtree (or binary tree, ternary tree) split when the coding mode is Inter.

[0113] For the example, the information on whether the adaptive partition candidate group structure according to the encoding / decoding information is supported may be explicitly generated or implicitly determined.

[0114] The examples described above describe situations where the segmentation mode and segmentation settings are determined based on encoding / decoding settings. These examples illustrate some situations based on various factors, and variations in other situations are possible. Furthermore, the segmentation mode and segmentation settings may be determined based on a combination of multiple factors. For example, the segmentation mode and segmentation settings may be determined based on block type, size, shape, encoding / decoding information, and the like.

[0115] In addition, factors related to the segmentation method, settings, etc. in the examples may be implicitly determined or information may be explicitly generated to determine whether the adaptive situation as in the above examples is allowed.

[0116] The segmentation depth in the segmentation setting indicates the number of times the space is segmented based on the initial block (the segmentation depth of the initial block in this example is 0). The larger the segmentation depth, the smaller the blocks can be segmented. This can be done by setting depth-related settings differently depending on the segmentation method. For example, in a tree-based segmentation method, the segmentation depth of a binary tree can share the same depth as that of a ternary tree, the segmentation depth of a quadtree can use a different depth from the segmentation depth of a binary tree, and so on. Different depths can be used depending on the type of tree.

[0117] When using different segmentation depths depending on the tree type in the example, the segmentation depth at the start position of the tree segmentation (the block before segmentation in this example) can be set to 0. The segmentation depth can be calculated centered on the position where segmentation starts, rather than based on the segmentation range of each tree (the maximum value in this example).

[0118] Figure 4 A block partitioning method based on a tree structure is shown as an embodiment to which the present invention is applied.

[0119] In the figure, thick solid lines represent basic coding blocks, thick dashed lines represent quadtree split boundaries, double solid lines represent symmetric binary tree split boundaries, solid lines represent ternary tree split boundaries, and thin solid lines represent asymmetric binary tree split boundaries. All lines except the thick solid lines represent boundaries based on the respective splitting methods. The splitting configuration described below (e.g., splitting type, splitting information, splitting information configuration order, etc.) is not limited to the example shown here and can be modified in various ways.

[0120] For ease of description, let's assume that the top left, top right, bottom left, and bottom right blocks (N×N; 64×64) based on the basic coding block (2N×2N; 128×128) have their own block split settings. First, let's assume that four sub-blocks have been obtained due to a single split operation (split depth 0->1; i.e., the split depth increased by 1) on the initial block, and that the quadtree split settings are such that the maximum coding block is 128×128, the minimum coding block is 8×8, and the maximum split depth is 4. These settings are common to all blocks.

[0121] (1st; upper left block; A1 to A6)

[0122] In this example, as a case of supporting single-tree segmentation (quadtree in this example), the size and shape of the blocks that can be obtained can be determined through a block segmentation setting, such as the maximum coding block, the minimum coding block, the segmentation depth, etc. In this example, as a case where the block that can be obtained according to the segmentation is one (2 segmentations are performed in the horizontal and vertical directions respectively), the segmentation information required for one segmentation operation (based on the 4M×4N block before segmentation, the segmentation depth is increased by 1) is a flag indicating whether to segment (if it is 0 in this example, it does not segment; if it is 1, it segments), the candidates that can be obtained are 4M×4N and 2M×2N.

[0123] (No. 2; upper right block; A7-A11)

[0124] This example supports multiple tree-based partitioning (quadtree and binary tree in this example), and the size and shape of the blocks that can be obtained can be determined by multiple block partitioning settings. In this example, it is assumed that the maximum coding block for the binary tree is 64×64, the minimum coding block is a length of 4, and the maximum partition depth is 4.

[0125] In this example, there are more than 2 blocks that can be obtained according to the segmentation (2 or 4 in this example). The segmentation information required for one segmentation operation (the quadtree segmentation depth increases by 1) is a flag indicating whether to segment, a flag indicating the segmentation type, a flag indicating the segmentation type, and a flag indicating the segmentation direction. The candidates that can be obtained can be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, and 3M×4N / M×4N.

[0126] If the quadtree and binary tree splitting ranges overlap (i.e., the range in which both quadtree splitting and binary tree splitting can be performed in the current step), and the current block (before splitting) is a block obtained by quadtree splitting (a block obtained by quadtree splitting in a parent block <when ​​the splitting depth is 1 less than the current one>), the following distinction can be made and splitting information can be configured. That is, when a block supported by each splitting setting can be obtained by multiple splitting methods, the following classification can be used to generate splitting information.

[0127] (1) Overlapping of quadtree partitioning and binary tree partitioning

[0128] Table 1

[0129] a b v d QT 1 No Split 0 0 SBT hor 0 1 0 ABT hor 1 / 4 0 1 0 0 ABT hor 3 / 4 0 1 0 1 SBT ver 0 1 1 ABT ver 1 / 4 0 1 1 0 ABT ver 3 / 4 0 1 1 1

[0130] In the table, a is a flag indicating whether to perform quadtree splitting. 1 indicates quadtree splitting (QT). If the flag is 0, b, which indicates whether to perform binary tree splitting, is checked. If b is 0, no splitting is performed on the block (No Split). If b is 1, binary tree splitting is performed.

[0131] c is a flag indicating the direction of the split. If c is 0, it indicates horizontal splitting (hor), and if c is 1, it indicates vertical splitting (ver). d is a flag indicating the splitting type. If d is 0, it indicates symmetric splitting (SBT), and if d is 1, it indicates asymmetric splitting (ABT). Only when d is 1, the detailed splitting ratio information (1 / 4 or 3 / 4) for asymmetric splitting is confirmed. When d is 0, in the left / right block or the upper / lower block, the left block has a ratio of 1 / 4 to the upper block, and the right block has a ratio of 3 / 4 to the lower block. If d is 1, the opposite is true.

[0132] (2) When only binary tree partitioning is possible

[0133] In the table, the division information can be represented by flags b to e other than a.

[0134] against Figure 4 The A7 block in the block A belongs to the case of generating segmentation information in (1), because quadtree segmentation can be performed in the blocks before segmentation (A7~A11) (that is, although quadtree segmentation can be performed, binary tree segmentation is performed instead of quadtree segmentation).

[0135] On the contrary, for A8 to A11, since the blocks before segmentation (A8 to A11) have been segmented by binary tree instead of quadtree (i.e., in the blocks<A8~A11> In the case where quadtree segmentation can no longer be performed), it belongs to the case of generating segmentation information in (2).

[0136] (3rd; lower left block; A12-A15)

[0137] This example supports multiple tree-based partitioning (quadtree, binary tree, and ternary tree in this example). Multiple block partitioning settings can be used to determine the size and shape of the blocks that can be obtained. In this example, it is assumed that the maximum coding block for the binary / ternary tree is 64×64, the minimum coding block is a length of 4, and the maximum partition depth is 4.

[0138] In this example, there are more than 2 blocks that can be obtained according to the segmentation (2, 3, and 4 in this example). The segmentation information required in one segmentation operation is a flag indicating whether to perform segmentation, a flag indicating the segmentation type, and a flag indicating the segmentation direction. The candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×2N / 4M×N, and M×4N / 2M×4N / M×4N.

[0139] If the quadtree and binary / ternary tree partition ranges overlap, and the current block is a block obtained by quadtree partition, the following situations can be used to distinguish and configure the partition information.

[0140] (1) Quadtree partitioning overlaps with binary / ternary tree partitioning

[0141] Table 2

[0142] a b c d QT 1 No Split 0 0 BT hor 0 1 0 0 TT hor 0 1 0 1 BT hor 0 1 1 0 TT ver 0 1 1 1

[0143] In the table, a is a flag indicating whether to perform quadtree splitting. If it is 1, quadtree splitting is performed. If the flag is 0, b, which indicates whether to perform binary or ternary tree splitting, is checked. If b is 0, no splitting is performed on the block. If b is 1, binary or ternary tree splitting is performed.

[0144] c is a flag indicating the direction of segmentation. If c is 0, it means horizontal segmentation. If c is 1, it means vertical segmentation. d is a flag indicating the segmentation classification. If d is 0, it means binary tree segmentation (BT). If d is 1, it means ternary tree segmentation (TT).

[0145] (2) Situations where only binary / ternary tree partitioning is possible

[0146] In the table, the division information can be represented by flags b to d in addition to a.

[0147] against Figure 4 The A12 and A15 blocks in the figure can be quadtree-partitioned in the blocks before partitioning (A12 to A15), so they belong to the case of generating sub-partition information in (1).

[0148] On the contrary, if A13 and A14 are blocks before segmentation (A13, A14) that have been segmented by a ternary tree instead of a quadtree, this is the case of generating segmentation information in (2).

[0149] (4th; lower left block; A16-A20)

[0150] This example supports multiple tree-based partitioning (quadtree, binary tree, and ternary tree in this example). Multiple block partitioning settings can be used to determine the size and shape of the blocks that can be obtained. In this example, it is assumed that the maximum coding block for the binary / ternary tree is 64×64, the minimum coding block is a length of 4, and the maximum partition depth is 4.

[0151] In this example, there are more than 2 blocks that can be obtained according to the segmentation (2, 3, and 4 in this example). The segmentation information required for one segmentation operation is a flag indicating whether to perform segmentation, a flag indicating the segmentation classification, a flag indicating the segmentation type, and a flag indicating the segmentation direction. The candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N, 4M×N / 4M×2N / 4M×N, M×4N / 2M×4N / M×4N.

[0152] If the quadtree and binary / ternary tree partition ranges overlap, and the current block is a block obtained by quadtree partition, the following situations can be used to distinguish and configure the partition information.

[0153] (1) Quadtree partitioning overlaps with binary / ternary tree partitioning

[0154] Table 3

[0155]

[0156] In the table, a is a flag indicating whether to perform quadtree splitting. If it is 1, quadtree splitting is performed. If the flag is 0, b, which indicates whether to perform binary tree splitting, is checked. If b is 0, no splitting is performed on the block. If b is 1, binary or ternary tree splitting is performed.

[0157] c is a flag indicating the split direction. If c is 0, it indicates horizontal splitting, and if c is 1, it indicates vertical splitting. d is a flag indicating the split type. If d is 0, it indicates ternary splitting, and if d is 1, it indicates binary splitting. When d is 1, e, the flag for the split type, is checked. If e is 0, symmetric splitting is performed, and if e is 1, asymmetric splitting is performed. When e is 1, detailed information about the split ratio in asymmetric splitting is confirmed, similar to the previous example.

[0158] (2) Situations where only binary / ternary tree partitioning is possible

[0159] In the table, the division information can be represented by flags b to f other than a.

[0160] because Figure 4The A20 block in the figure is a case where quadtree partitioning can be performed on the blocks before partitioning (A16 to A19), which belongs to the case of generating sub-partition information in (1).

[0161] On the contrary, for A16 to A19, the case where binary tree partitioning rather than quad tree partitioning has been performed in the pre-partition blocks (A16 to A19) belongs to the case of generating partition information in (2).

[0162] Figure 5 The process of performing intra-frame prediction on the current block based on a matrix as an embodiment of the present invention is shown.

[0163] refer to Figure 5 , an intra prediction mode for intra prediction of a current block may be determined ( S500 ).

[0164] When performing intra-frame prediction on a current block, the encoding / decoding apparatus may determine an intra-frame prediction mode. The current block may be a coding block (CU), a prediction block (PU), a transform block (TU), or any one of these blocks.

[0165] (Embodiment 1) The intra-frame prediction mode can be determined based on the information of the transmitted signal. The information can specify any one of the N intra-frame prediction modes predefined in the encoding / decoding device. The predefined intra-frame prediction mode represents all intra-frame prediction modes that can utilize the current block, and N can be a natural number less than or equal to 67 and greater than or equal to 11 (for example, 67, 35, 11). In addition, the value of N can be determined based on the size of the current block. For example, when the current block is smaller than 8×8, N is determined to be 35, otherwise, N can be determined to be any one of 19 or 11.

[0166] (Embodiment 2) The intra-frame prediction mode may also be determined by a default mode or index pre-agreed in the encoding / decoding device. The default mode may be at least one of Planar mode (index 0), DC mode (index 1), horizontal mode (index 18), vertical mode (index 50), and diagonal mode (indexes 2, 34, and 66). The index corresponds to the case where there are 67 pre-defined intra-frame prediction modes, and each mode may be assigned a different index based on the value of N.

[0167] (Embodiment 3) The intra-frame prediction mode can be variably determined based on coding information. The coding information may include not only information encoded and transmitted in the coding device, but also information derived based on the information transmitted in the decoding device. The coding information may be information about at least one of the current block and the adjacent blocks. The adjacent blocks include spatial and / or temporal adjacent blocks of the current block. The spatial adjacent blocks may represent blocks adjacent to at least one of the left side, upper layer, upper left layer, lower left layer, or upper right layer of the current block.

[0168] The encoding information may include block size / shape, block availability, partition type, number of partitions, component type, prediction mode, information regarding intra-frame prediction mode, inter-frame mode, motion information, transform type, transform skip mode, information regarding non-zero residual coefficients, scanning order, color format, loop filter information, etc. The block size may be represented by either width or height, the minimum / maximum of the width and height, the sum of the width and height, the number of samples belonging to the block, etc. The block availability may be determined by considering the block position, the range of the parallel processing area, the decoding order, etc. The prediction mode may indicate information indicating intra-frame mode or inter-frame mode. The intra-frame prediction mode information may include information regarding whether the intra-frame prediction mode is non-directional, whether the intra-frame prediction mode is vertical / horizontal, the directionality of the intra-frame prediction mode, and the number of intra-frame prediction modes predefined in the encoding / decoding device. The inter-frame mode may indicate information indicating merge / skip mode, AMVP mode, or current picture reference mode. The current picture reference mode indicates the method of predicting the current block using a reconstructed area of ​​the current picture. The current image may be the image to which the current block belongs. The current image may be added to a reference image list for inter-frame prediction, and the current image may be arranged after a short-term reference image or a long-term reference image in the reference image list. The motion information may include a prediction direction flag, a motion vector, a reference image index, etc.

[0169] (Embodiment 4) The intra prediction mode may also be derived based on an MPM list and an MPM index. The MPM list includes multiple MPMs, and the MPM may be determined based on the intra prediction modes of spatially / temporally neighboring blocks of the current block. The number of MPMs is x, where x may be 3, 4, 5, 6, or a larger integer.

[0170] For example, the MPM list may include at least one of the intra prediction modes mode A, (mode An), (mode A+n), or a default mode of the adjacent block. The value of n may be 1, 2, 3, 4, or a larger integer. The adjacent block may represent a block adjacent to the left and / or upper layer of the current block. The default mode may be at least one of a Planar mode, a DC mode, or a predetermined directional mode. The predetermined directional mode may include at least one of a horizontal mode (mode V), a vertical mode (mode H), (mode Vk), (mode V+k), (mode Hk), or (mode H+k).

[0171] The MPM index may specify an MPM in the MPM list that is the same as the intra prediction mode of the current block. That is, the MPM specified by the MPM index may be set as the intra prediction mode of the current block.

[0172] Any one of the aforementioned embodiments 1 to 4 may be selectively used to determine the intra-frame prediction mode of the current block, and the intra-frame prediction mode of the current block may be determined based on a combination of at least two of the embodiments 1 to 4. A predetermined flag may be used for the selection, and in this case, the flag may be encoded by the encoding device and transmitted as a signal.

[0173] refer to Figure 5 , a reference sample for intra prediction of a current block may be determined ( S510 ).

[0174] The reference sample may be derived from a neighboring region of the current block, wherein the neighboring region of the current block may include at least one of a left side, a right side, an upper layer, a lower left layer, an upper left layer, a lower right layer, or an upper right layer of the current block.

[0175] The adjacent area may include one or more sample lines. Specifically, the number of sample lines belonging to the adjacent area is k, where k can be 1, 2, 3, 4, or a natural number greater than these. The k value can be a fixed value pre-agreed in the encoding / decoding device, or can be variably determined based on the aforementioned encoding information. For example, when the current block is of a first size (e.g., 4×4, 4×8, 8×4), the adjacent area can be configured as 1 sample line, and when the current block is of a second size (e.g., 8×8, 16×16, etc.), the adjacent area can be configured as 2 sample lines. The sample line can be determined in a vertical direction or a horizontal direction according to the position of the adjacent area. In addition, the sample line can be in contact with the current block, or can be away from a predetermined distance in the vertical and / or horizontal direction based on the current block.

[0176] The multiple sample lines may be continuous in the vertical and / or horizontal directions relative to the current block, or may be separated from each other by a predetermined distance. In one embodiment, when there are two sample lines above the current block, the sample lines from the lowest of the two lines upward are named the first and second sample lines, respectively. In this case, the first and second sample lines may be in contact with each other or separated by a predetermined distance. The predetermined distance may be represented by i line lengths (i.e., width or height). i may be a natural number of 0, 1, 2, 3, or greater. In one embodiment, when there are three sample lines above the current block, the sample lines from the lowest of the two lines upward are named the first, second, and third sample lines, respectively. In this case, the first sample line may be in contact with the second sample line, and the second sample line may be in contact with the third sample line. Alternatively, the first through third sample lines may be separated by the predetermined distances. In this case, the interval (d1) between the first and second sample lines may be the same as the interval (d2) between the second and third sample lines. Alternatively, d1 can be set to be greater than d2, or conversely, d1 can be set to be less than d2. As an embodiment, when there are four or more sample lines in the upper layer of the current block, the four sample lines can be determined in the same manner as in the case of three sample lines. In addition, this embodiment is applicable not only to sample lines located in the upper layer, but also to sample lines located on the left side. The detailed description is omitted here.

[0177] The reference samples may be derived by utilizing all or part of the samples belonging to the neighboring regions.

[0178] (Example 1) Some samples of the adjacent area may be samples at pre-agreed positions in the encoding / decoding device. The pre-agreed positions may include: at least one of the leftmost sample, the rightmost sample, or the middle sample of the upper sample line. The pre-agreed positions may include: at least one of the uppermost sample, the lowermost sample, or the middle sample of the left sample line. Alternatively, the pre-agreed positions may include: at least one of the odd-numbered samples of the upper and / or left sample lines, or at least one of the odd-numbered samples. Alternatively, the pre-agreed positions may also include: samples with x-coordinates that are multiples of j in the samples of the upper sample line, or samples with y-coordinates that are multiples of j in the samples of the left sample line. Wherein, j may be a natural number of 2, 3, 4, or a larger number.

[0179] (Embodiment 2) Some samples of the adjacent region may also be variably determined based on the coding information, wherein the coding information is as described above, and detailed description thereof is omitted here.

[0180] Either one of the embodiments 1 or 2 may be selectively used, or some samples may be specified based on a combination of embodiments 1 and 2. At this time, as described above, the intervals between some samples may be set identically, but this is not limiting, and the intervals between some samples may also be set differently.

[0181] The number of some samples may be 1, 2, 3, 4 or more predefined in the encoding / decoding device. In addition, the number of some samples may be defined differently for the left adjacent area and the upper adjacent area of ​​the current block, respectively. For example, when the width of the current block is greater than the height, the number of some samples belonging to the upper adjacent area (numSamA) may be greater than the number of some samples belonging to the left adjacent area (numSamL). On the contrary, when the width of the current block is less than the height, numSamA may be less than numSamL. Alternatively, the number of some samples may be variably determined based on the aforementioned encoding information.

[0182] The samples of the adjacent region may be predicted samples or reconstructed samples. The predicted samples may be obtained by intra-frame prediction or inter-frame prediction. The reconstructed samples may be reconstructed samples before or after the loop filter is applied.

[0183] On the other hand, the reference sample can be derived directly from the sample of the adjacent area (CASE 1), or can be derived by downsampling the samples of the adjacent area (CASE 2). Any one of CASE 1 and CASE 2 can be selectively used. The selection can be made based on the aforementioned encoding information. For example, when the size of the current block is less than a predetermined threshold, the reference sample can be derived based on CASE 1, otherwise, the reference sample can be derived based on CASE 2. The size can be represented by any one of the width, height, maximum / minimum value of width and height, ratio of width to height, or product of width and height of the current block. As an example, when the current block is smaller than 8×8, the reference sample can be derived from the sample of the adjacent area, otherwise, the reference sample can be derived by downsampling the samples of the adjacent area. For the downsampling method, by referring to Figure 6 and Figure 7 Learn more.

[0184] refer to Figure 5 , a matrix for matrix-based intra prediction may be determined ( S520 ).

[0185] The matrix can be determined based on at least one of the intra-frame prediction mode determined in step S500 or the size of the current block. Alternatively, the matrix can be determined by limiting consideration to only the intra-frame prediction mode of the current block, or limiting consideration to only the size of the current block. The size can be represented by either width or height, the minimum / maximum value of the width and height, the sum of the width and height, the number of samples belonging to the current block, etc. However, this is not limiting, and the matrix can be determined by further considering encoding information about the current block. The encoding information is as described above, and a detailed description thereof is omitted here.

[0186] Specifically, the matrices pre-agreed in the encoding / decoding device can be divided into multiple matrix groups. The multiple matrix groups can be configured by a first matrix group, a second matrix group, ..., an mth matrix group. Wherein, m can be a natural number of 2, 3, 4, 5 or more. Based on the size of the current block, the current block can selectively utilize any one of the multiple matrix groups. For example, when the size of the current block is 4×4, the first matrix group can be utilized, when the size of the current block is 8×4, 4×8 and 8×8, the second matrix group can be utilized, and in other cases, the third matrix group can be utilized. The matrix group selected based on the size of the current block may include one or more matrix candidates. Any one of the multiple matrix candidates can be determined by the matrix of the current block. The determination can be made based on the encoding information of the current block (e.g., intra-frame prediction mode).

[0187] The number of the pre-agreed matrices may be the same as the number of the aforementioned pre-defined intra-frame prediction modes. In addition, the number of the pre-agreed matrices may be less than the number of the pre-defined intra-frame prediction modes. In this case, one matrix may match multiple intra-frame prediction modes. For example, one matrix may match two intra-frame prediction modes. In this case, the number of the pre-agreed matrices may have a value of 1 / 2 times the number of pre-defined intra-frame prediction modes. However, this is not limited to this, and the number of intra-frame prediction modes that match one matrix may be 3, 4, 5, 6, or more.

[0188] As an embodiment, the matching may be determined by considering the directionality and / or symmetry of the intra prediction mode.

[0189] The predefined intra-frame prediction mode may include a directional mode having a predetermined angle. The directional mode may be divided into a first mode group having horizontal directionality and a second mode group having vertical directionality. Assuming that the number of directional modes is 65, the first mode group may be configured to belong to a mode between index 2 and index 34, and the second mode group may be configured to belong to a mode between index 34 and index 66.

[0190] The encoding / decoding device only defines the matrix for the first mode group, and the second mode group can also use the matrix defined for the first mode group. Conversely, the encoding / decoding device only defines the matrix for the second mode group, and the first mode group can also use the matrix defined for the second mode group. In this case, the number of pre-agreed matrices can have a value of 1 / 2 times the number of pre-defined intra-frame prediction modes. As an embodiment, when the number of symmetrical mode groups is x, the number of pre-agreed matrices can have a value of 1 / x times the number of pre-defined intra-frame prediction modes. Wherein, x can be 3, 4 or more.

[0191] The symmetry can be based on the intra-frame prediction mode with an angle of -45°, including the symmetry of the prediction angle between the mode with vertical directivity and the mode with horizontal directivity. Wherein, the intra-frame prediction mode with directionality has a prediction angle (PredAngle) according to each directionality. Wherein, the mode with vertical directivity can be based on the intra-frame prediction mode with an angle of -45°, including a mode with an angle of -45°<(PredAngle)≤45° from the mode and the mode along the x-axis. Wherein, the mode with horizontal directivity can be based on the intra-frame prediction mode with an angle of -45°, including a mode other than the mode with an angle of -45°<(PredAngle)≤45° from the mode along the y-axis.

[0192] refer to Figure 5 , the current block can be predicted based on the reference sample and the matrix (S530).

[0193] When the reference sample is determined in step S510 and the matrix is ​​determined in step S520, the encoding / decoding apparatus may predict the current block based on the reference sample and the matrix.

[0194] The step of predicting the current block may include applying the matrix to the reference samples to obtain prediction samples of the DS block (hereinafter referred to as first prediction samples). The DS block may represent the current block or a downsampled current block. That is, the DS block may have the same size as the current block, or the size of the current block may be 1 / 2, 1 / 4, 1 / 8, or 1 / 16 of (at least one of) its width or height. For example, when the current block is a 4×4, 4×8, or 8×4 block, the DS block may be a 4×4 block. Alternatively, when the current block is an 8×8, 8×16, or 16×8 block, the DS block may be a 4×4 or 8×8 block. Alternatively, when the current block is greater than or equal to 16×16, the DS block may be an 8×8 or 16×16 block. However, DS blocks are not limited to square blocks and may also be non-square blocks. Alternatively, DS blocks may also be limited to square blocks. Applying the matrix may include multiplying the reference samples by weight values ​​obtained from the matrix.

[0195] The step of obtaining the first prediction sample may include at least one of a step of adding an offset value or a filtering step.

[0196] The step of obtaining the first prediction samples may further include the step of rearranging the first prediction samples, and the rearrangement may be performed only when multiple intra-frame prediction modes are matched in one matrix.

[0197] Alternatively, the re-arrangement may be performed when the intra prediction mode of the current block belongs to the first mode group having horizontal directivity. For example, when the intra prediction mode of the current block belongs to the first mode group having horizontal directivity, the re-arrangement is performed for the first prediction samples of the DS block, and when the intra prediction mode of the current block belongs to the second mode group having vertical directivity, the re-arrangement may not be performed for the first prediction samples of the DS block.

[0198] On the contrary, when the intra prediction mode of the current block belongs to the first mode group with vertical directivity, the rearrangement may be performed. For example, when the intra prediction mode of the current block belongs to the first mode group with horizontal directivity, the first prediction samples of the DS block are not rearranged. When the intra prediction mode of the current block belongs to the second mode group with vertical directivity, the first prediction samples of the DS block may be rearranged.

[0199] The rearrangement may be performed as shown in the following formula 1. Here, x may represent an x-axis coordinate value, and y may represent a y-axis coordinate value. That is, the rearrangement may represent a process of allocating the first prediction sample of the (x, y) coordinate to the (y, x) coordinate.

[0200] [Formula 1]

[0201] First prediction sample [x][y] = First prediction sample [y][x]

[0202] Alternatively, the rearrangement according to the present invention may also represent a process of rotating the DS block composed of the first prediction samples by a predetermined angle, wherein the predetermined angle may represent 90 degrees or 180 degrees in a clockwise direction or 90 degrees or 180 degrees in a counterclockwise direction.

[0203] The step of predicting the current block may further include the step of up-sampling the current block based on at least one of adjacent reconstructed samples or the first predicted samples to obtain a second predicted sample.

[0204] During the upsampling process, whether to perform the upsampling or at least one of the upsampling methods may be determined based on the encoding information of the current block. For example, whether to perform the upsampling or at least one of the upsampling methods may be determined based on the size of the DS block consisting of the first prediction samples and the size of the current block. The block size may be represented by either width or height, the minimum / maximum value of the width and height, the sum of the width and height, the number of samples belonging to the block, etc.

[0205] Whether to perform the upsampling may be determined only when the size of the DS block composed of the first prediction samples is smaller than the size of the current block.

[0206] The upsampling method may include: allocating the first prediction samples to predetermined positions within the current block using a ratio of the size of the DS block composed of the first prediction samples to the size of the current block; and interpolating a remaining area within the current block. The remaining area may refer to an area in the current block other than the area where the first prediction samples are divided. Figures 8 to 10 The division of the first prediction samples and the interpolation method for the remaining area are described in detail.

[0207] Figure 6 A method for determining reference samples by downsampling adjacent areas is shown as an embodiment applicable to the present invention.

[0208] refer to Figure 6 , Figure 6 (a) shows a case where the adjacent region used for intra prediction is located to the left and above the current block. Furthermore, as an embodiment, the sample line located to the left of the current block is in contact with the current block and consists of one sample line in the vertical direction. The sample line located above the current block is in contact with the current block and consists of one sample line in the horizontal direction.

[0209] The reference sample may include a down-sampled region formed by down-sampling an adjacent region of the current block.

[0210] The downsampled region may be derived from an average value, a maximum value, a minimum value, a mode value, or a filtered value of all or some samples belonging to the neighboring region.

[0211] When derived from the average value, the down-sampling region may be formed by allocating average values ​​of different N samples to the samples in the down-sampling region.

[0212] The N different samples can be arranged consecutively or spaced apart by a certain interval. The certain interval is an interval of one or more sample sizes. When there are multiple intervals, the multiple intervals can be uniform or non-uniform. (Wherein, N is greater than 2 and less than the total number of samples belonging to the adjacent region.) In addition, the combination of the N different samples is referred to as a sample group. In this case, the first sample group may overlap with the second sample group, or may not overlap.

[0213] As an example, Figure 6 It is shown that N is 2, the two sample groups do not overlap with each other, and the average value of the two samples belonging to each sample group is respectively allocated to one sample in the downsampling area to perform downsampling.

[0214] Alternatively, three consecutive samples (S1, S2, S3) may constitute a first sample group, and the average value of the three samples belonging to the first sample group may be assigned to the sample (DS1) in the downsampled area. Three consecutive samples (S2, S3, S4) may constitute a second sample group, and the average value of the three samples belonging to the second sample group may be assigned to the sample (DS2) in the downsampled area.

[0215] Alternatively, after determining the minimum or maximum value of the two samples (S1 and S2) belonging to the first sample group, it can be assigned to the sample (DS1) in the downsampled area. Similarly, after determining the minimum or maximum value of the two samples (S3 and S4) belonging to the second sample group, it can be assigned to the sample (DS2) in the downsampled area. This can also be used in the case where the first / second sample group consists of three samples.

[0216] Alternatively, in the upper adjacent region, samples at predefined positions among the multiple samples belonging to the first sample group can be allocated to the samples (DS1) in the downsampling region, and samples at predefined positions among the multiple samples belonging to the second sample group can be allocated to the samples (DS2) in the downsampling region. The predefined position can represent a fixed position pre-agreed in the encoding / decoding device, and as an example, can be any one of the leftmost, rightmost, or middle positions. In the left adjacent region, samples at predefined positions among the multiple samples belonging to each sample group can also be respectively allocated to the samples in the downsampling region. In this case, the predefined position can be any one of the uppermost, lowermost, or middle positions.

[0217] Figure 7 A downsampling method based on weighted average is shown as an embodiment to which the present invention is applicable.

[0218] In this embodiment, the average value can be calculated by the following formula (hereinafter referred to as the first average expression):

[0219] the samples and / or number of samples belonging to the sample group;

[0220] Or it can be calculated by the following formula (hereinafter referred to as the second average expression):

[0221] Sum (weighted value × samples belonging to the sample group) / number of samples.

[0222] Figure 7 (a) shows the case where the aforementioned sample group consists of 3 samples. In this case, the weighted values ​​applied to the 3 samples can be determined to be in a ratio of 1:2:1. Figure 7 As shown in (b), when the sample group consists of 5 samples, the weighted values ​​can be determined as a ratio of 1:1:4:1:1. Figure 7 As shown in (c), when the sample group consists of 6 samples, the weighted values ​​can be determined as 1:2:1:2:2:1 or 1:2:2:1:2:1 in the Z direction with the upper layer on the left as the starting point. In addition, the Figure 7 Although (a) and (c) show the weighting values ​​applicable to the upper adjacent area, this can also be applied to the left adjacent area.

[0223] The average value may also include a result value derived by applying a predetermined operation to multiple average values ​​calculated using the first average expression or the second average expression. The predetermined operation may be the first average expression or the second average expression described above. For example, if three samples (i.e., the first to third samples) belong to a sample group, the average value (first value) between the first sample and the second sample, and the average value (second value) between the second sample and the third sample, may be calculated, respectively. The average value may be derived from the average value between the calculated first and second values.

[0224] The aforementioned downsampling method can be applied only to the upper adjacent area, or conversely, only to the left adjacent area. Figure 6 The downsampling method (hereinafter referred to as the first method) can be applied to any one of the upper layer or the left adjacent area, according to Figure 7 The downsampling method (hereinafter referred to as the second method) can be applied to another one in the upper layer or the left adjacent area.

[0225] In addition, considering the size / shape of the current block, at least one of the first method or the second method can be selectively utilized. For example, when the width of the current block is greater than a predetermined threshold, the first method can be applied to the upper adjacent area of ​​the current block; otherwise, the second method can be applied. The height of the current block can also be downsampled in the same manner. Alternatively, when the current block is non-square, the first method can be applied to either the upper or left adjacent area, and the second method can be applied to the other area. In this case, when the width of the current block is greater than the height, the first method can be applied to the upper adjacent area, and the second method can be applied to the left adjacent area. Conversely, when the width of the current block is less than the height, the second method can be applied to the upper adjacent area, and the first method can be applied to the left adjacent area. When the current block is square, the same downsampling method can be used in the upper and left adjacent areas, where the downsampling method can also be limited to the first method.

[0226] Figure 8 The diagram shows a method for dividing the first prediction sample and interpolating the remaining areas as an embodiment applicable to the present invention.

[0227] refer to Figure 8 In (a), the prediction samples of the DS block can be allocated to the prediction samples at a predetermined position in the current block. The predetermined position can be determined by considering the size ratio between the current block and the DS block. For example, the correspondence relationship between the prediction samples of the DS block and the current block can be defined as follows:

[0228] [Formula 2]

[0229] First prediction sample curBLK[(x+1)×r-1][(y+1)×r-1]=first prediction sample dsBLK[x][y]

[0230] Where r represents the size ratio between the current block and the DS block, x and y are the x-axis and y-axis coordinates of the first predicted sample in each DS block, respectively. The first predicted sample curBLK can represent the position of the first predicted sample in the current block, and the first predicted sample dsBLK can represent the position of the first predicted sample in the DS block.

[0231] The interpolation, reference Figure 8 In (b), samples in the current block in which the first prediction sample is not partitioned (hereinafter referred to as interpolation target samples) may be derived using at least one of the reconstructed samples adjacent to the first prediction sample partitioned toward the current block or the current block (hereinafter referred to as interpolation reference samples). Furthermore, the interpolation reference samples may further include prediction samples generated by interpolation prior to the current interpolation target sample (i.e., the previous interpolation target sample).

[0232] The positions and ranges of the reconstructed samples adjacent to the current block are the same as those of the aforementioned reference samples, so their detailed description is omitted here.

[0233] Depending on the position of the interpolation target sample, the interpolation reference sample may be composed of multiple first prediction samples, or may be composed of at least one first prediction sample and at least one reconstructed adjacent sample. The reconstructed adjacent sample may selectively utilize any one of the samples having the same x-coordinate or y-coordinate as the interpolation target sample, or may utilize multiple samples having at least one of the x-coordinate or y-coordinate identical to the interpolation target sample. The selection may be made based on the position of the interpolation target sample. For example, if the interpolation target sample has the same x-coordinate as the first prediction sample, the reconstructed adjacent samples may only include samples having the same x-coordinate as the interpolation target sample. Conversely, if the interpolation target sample has the same y-coordinate as the first prediction sample, the reconstructed adjacent samples may only include samples having the same y-coordinate as the interpolation target sample. Alternatively, the reconstructed adjacent samples may include multiple samples located on the same horizontal and vertical lines as the interpolation target sample.

[0234] The interpolation target sample may be derived from representative values ​​of a plurality of interpolation reference samples, wherein the representative value may include any one of an average value, a minimum value, a maximum value, a mode, or a median value.

[0235] The average value can be calculated by the following formula (hereinafter referred to as the first average expression):

[0236] Total difference reference samples / number of difference reference samples;

[0237] Or it can be calculated by the following formula (hereinafter referred to as the second average expression):

[0238] Sum(weighted value × difference reference sample) / number of difference reference samples.

[0239] The weighted value according to the second average expression may be determined based on the relative / absolute distance between the interpolation object sample and the interpolation reference sample, which may refer to Figure 9 Detailed description.

[0240] Figure 9 As one embodiment to which the present invention is applicable, a method of assigning a weight value to a distance in an interpolation step is shown.

[0241] The weighted value according to the present invention may include a weighted value determined based on the distance from the interpolation object sample to the interpolation reference sample. As an embodiment, the reference Figure 9 If the first interpolation target sample 910 is interpolated, since the distance ratio from the first interpolation target sample 910 to the first interpolation reference sample 911 and the second interpolation reference sample 912 is 3:1, the weighting ratio applied to the first interpolation reference sample 911 and the second interpolation reference sample may be 1:3. If the second interpolation target sample 920 is interpolated, since the distance ratio from the second interpolation target sample 920 to each of the first interpolation reference sample 921 and the second interpolation reference sample 922 is 1:1, the weighting ratio applied to the first interpolation reference sample 921 and the second interpolation reference sample 922 may be 1:1.

[0242] In addition, the interpolation filter according to the present invention may have directionality, which may include vertical, horizontal, Z-shaped, diagonal, and the like.

[0243] The interpolation may be performed based on a predetermined priority order. The priority order may be either: performing interpolation in the vertical direction first and then in the horizontal direction (a first priority order), or performing interpolation in the horizontal direction first and then in the vertical direction (a second priority order). Alternatively, interpolation may be performed in both the vertical and horizontal directions (a third priority order).

[0244] The current block can be interpolated using only any one of the first to third orders, or can be interpolated using a combination of at least two of the first to third orders. For the interpolation order, please refer to Figure 10 Learn more.

[0245] Figure 10 The sequence of interpolation steps is shown as one embodiment to which the present invention is applicable.

[0246] Figure 10(a) is about Figure 9 Specifically, the vertical line to which the first prediction sample belongs may be interpolated first, and then the horizontal line may be interpolated based on the interpolated line and the interpolated reference sample on the left side of the current block.

[0247] Figure 10 (b) is about Figure 9 Specifically, the horizontal line to which the first prediction sample belongs may be interpolated first, and then the vertical line may be interpolated based on the interpolated line and the interpolation reference sample of the upper layer of the current block.

[0248] Figure 10 (c) is about Figure 9 The third order in . First, the vertical and horizontal lines to which the first prediction sample belongs can be interpolated. Then, interpolation is performed on the remaining samples that have not been interpolated. In this case, only the vertical lines or the horizontal lines can be interpolated, or both vertical and horizontal lines can be interpolated at the same time. If the vertical and horizontal lines are interpolated at the same time, an interpolation object sample can have a first interpolation value on the vertical line and a second interpolation value on the horizontal line. At this time, a representative value between the first interpolation value and the second interpolation value can be assigned to the interpolation object sample. The representative value can be derived from the average value, the minimum value, the maximum value, the mode, or the median value.

[0249] The interpolation order may be a pre-agreed order in the encoding / decoding device, or may be selectively determined based on the encoding information of the current block.

[0250] The order may be determined based on the block size, which may be expressed as either width or height, minimum / maximum of width and height, the sum of width and height, the number of samples belonging to the block, etc.

[0251] For example, when the size of the current block is greater than a predetermined threshold, the first interpolation may be performed; otherwise, the second interpolation may be performed. Conversely, when the size of the current block is less than a predetermined threshold, the second interpolation may be performed; otherwise, the first interpolation may be performed. The threshold may be a natural number of 8, 16, 32, or greater.

Claims

1. A video signal processing method, characterized in that: The video signal processing method is applied to a decoder, comprising: determining an intra prediction mode for a current block; Determining a reference sample for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode, the matrix being determined from a plurality of matrix groups based on the size and shape of the current block; and generating a prediction block by applying the matrix to the reference sample; wherein applying the matrix to the reference sample comprises multiplying the reference sample by a weight value obtained from the matrix; All or part of the generated prediction samples of the prediction block are transposed.

2. The video signal processing method according to claim 1, wherein: The determining of the reference sample comprises: Determining a neighboring area of ​​the current block; and Downsample the identified adjacent regions.

3. The video signal processing method according to claim 2, wherein: The adjacent areas are divided into multiple sample groups, The sample group consists of one or more samples. The representative value of the sample group is determined as the reference sample, The representative value is any one of an average value, a minimum value, a maximum value, a mode, or a median value.

4. The video signal processing method according to claim 1, wherein: The predicting the current block further includes: The current block is interpolated based on at least one of the prediction block or reconstructed samples adjacent to the current block.

5. A video signal processing method, characterized in that: The video signal processing method is applied to an encoder, comprising: determining an intra prediction mode for a current block; Determining a reference sample for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode, the matrix being determined from a plurality of matrix groups based on the size and shape of the current block; and generating a prediction block by applying the matrix to the reference sample; wherein applying the matrix to the reference sample comprises multiplying the reference sample by a weight value obtained from the matrix; All or part of the generated prediction samples of the prediction block are transposed.

6. The video signal processing method according to claim 5, wherein: The determining of the reference sample comprises: Determining a neighboring area of ​​the current block; and Downsample the identified adjacent regions.

7. The video signal processing method according to claim 6, wherein: The adjacent areas are divided into multiple sample groups, The sample group consists of one or more samples. The representative value of the sample group is determined as the reference sample, The representative value is any one of an average value, a minimum value, a maximum value, a mode, or a median value.

8. The video signal processing method according to claim 5, wherein: The predicting the current block further includes: The current block is interpolated based on at least one of the prediction block or reconstructed samples adjacent to the current block.

9. A decoder, characterized in that include: A first determining unit, configured to determine an intra prediction mode of a current block; Determining a reference sample for intra prediction of the current block; Based on the intra prediction mode, determining a predetermined matrix, the matrix being determined from a plurality of matrix groups based on the size and shape of the current block; and generating a prediction block by applying the matrix to the reference sample; wherein applying the matrix to the reference sample comprises multiplying the reference sample by a weight value obtained from the matrix; and transposing all or part of the prediction samples of the generated prediction block.

10. The decoder according to claim 9, characterized in that The first determining unit is configured to determine an adjacent area of ​​the current block; and downsample the determined adjacent area.

11. The decoder according to claim 10, characterized in that The adjacent areas are divided into multiple sample groups, The sample group consists of one or more samples. The representative value of the sample group is determined as the reference sample, The representative value is any one of an average value, a minimum value, a maximum value, a mode, or a median value.

12. The decoder according to claim 9, wherein: The first determining unit is configured to interpolate the current block based on the prediction block or at least one of the reconstructed samples adjacent to the current block.

13. An encoder, characterized in that include: A second determining unit, configured to determine an intra prediction mode of a current block; Determining a reference sample for intra prediction of the current block; Based on the intra prediction mode, determining a predetermined matrix, the matrix being determined from a plurality of matrix groups based on the size and shape of the current block; and generating a prediction block by applying the matrix to the reference sample; wherein applying the matrix to the reference sample comprises multiplying the reference sample by a weight value obtained from the matrix; and transposing all or part of the prediction samples of the generated prediction block.

14. The encoder according to claim 13, wherein The second determining unit is configured to determine an adjacent area of ​​the current block; and downsample the determined adjacent area.

15. The encoder according to claim 14, characterized in that The adjacent areas are divided into multiple sample groups, The sample group consists of one or more samples. The representative value of the sample group is determined as the reference sample, The representative value is any one of an average value, a minimum value, a maximum value, a mode, or a median value.

16. The encoder according to claim 13, wherein The second determining unit is configured to interpolate the current block based on at least one of the prediction block or reconstructed samples adjacent to the current block.

17. A computer-readable storage medium, wherein: Executable instructions are stored, and when the executable instructions are executed by one or more processors, the processors execute the video signal processing method according to any one of claims 1 to 4 or 5 to 8.

Images