Image encoding / decoding method and device using intra prediction

Abstract

To provide a video signal processing method and apparatus for improving encoding / decoding efficiency by matrix-based intra prediction.SOLUTION: An intra prediction method according to a video signal processing method of the present invention includes: determining an intra prediction mode of a current block; determining a reference sample used for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode; and predicting the current block based on the reference sample and the matrix.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image encoding / decoding method and device. [Background technology]

[0002] With the spread of the Internet, mobile terminals, and the growth of information and communication technology, the use of multimedia data is increasing rapidly. Therefore, the demand for the performance and efficiency of image processing systems to perform various services or tasks through image prediction in various systems is increasing significantly, but the research and development results that can meet this situation are quite lacking.

[0003] Therefore, there is a need to improve image processing, and in particular to improve image encoding or decoding performance, in prior art image encoding / decoding methods and apparatus. Summary of the Invention [Problem to be solved by the invention]

[0004] It is desirable that the present invention improves encoding / decoding efficiency by adaptive block division.

[0005] The present invention preferably improves encoding / decoding efficiency by matrix-based intra prediction.

[0006] The present invention provides a method and apparatus for determining reference samples and matrices used in matrix-based intra prediction.

[0007] The present invention provides a downsampling and upsampling method and apparatus for use in matrix-based intra prediction. [Means for solving the problem]

[0008] The video signal processing method and apparatus according to the present invention can determine an intra prediction mode of a current block, determine reference samples to be used for intra prediction of the current block, determine a predetermined matrix based on the intra prediction mode, and predict the current block based on the reference samples and the matrix.

[0009] In the video signal processing method and apparatus according to the present invention, the step of determining the reference sample may include the steps of determining a neighboring region of the current block, and performing downsampling on the determined neighboring region.

[0010] In the video signal processing method and apparatus according to the present invention, the adjacent region is divided into a plurality of sample groups, each of the sample groups consisting of one or more samples, and a representative value of the sample group is determined by the reference sample, and the representative value may be any one of an average value, a minimum value, a maximum value, a mode value or a median value.

[0011] In the video signal processing method and apparatus according to the present invention, the matrix may be determined by further taking into account coding information of the current block, and the coding information may include the size, shape, angle or direction of an intra prediction mode of the current block.

[0012] In the video signal processing method and apparatus according to the present invention, predicting the current block may include generating a predicted block by applying the matrix to the reference samples.

[0013] In the video signal processing method and apparatus according to the present invention, the step of predicting the current block may further include rearranging all or a part of prediction samples of the generated prediction block.

[0014] In the video signal processing method and apparatus according to the present invention, the step of predicting a current block may further include performing interpolation on the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block. Effect of the Invention

[0015] According to the present invention, the efficiency of encoding / decoding can be improved by dividing a tree block.

[0016] According to the present invention, the encoding / decoding efficiency can be improved by matrix-based intra prediction.

[0017] According to the present invention, the downsampling or upsampling used in the matrix-based intra prediction can improve the encoding / decoding efficiency. [Brief description of the drawings]

[0018] [Figure 1] 1 is a schematic block diagram illustrating an encoding device according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a schematic block diagram illustrating a decoding device according to an embodiment of the present invention. [Diagram 3] FIG. 13 is a diagram showing block division types according to an embodiment to which the present invention can be applied. [Figure 4] FIG. 2 is a diagram showing a block division method based on a tree structure according to an embodiment to which the present invention can be applied. [Diagram 5] 13 is a diagram illustrating a process of performing intra prediction on a current block based on a matrix according to an embodiment of the present invention. [Figure 6] FIG. 13 is a diagram illustrating a method for determining a reference sample by performing downsampling on neighboring blocks according to an embodiment to which the present invention can be applied. [Figure 7] FIG. 2 illustrates a weighted average based downsampling method according to an embodiment to which the present invention can be applied. [Figure 8]FIG. 11 is a diagram illustrating a method for dividing a first predicted sample and performing interpolation for other regions according to an embodiment to which the present invention can be applied. [Figure 9] FIG. 13 illustrates the assignment of weight values ​​to distances in an interpolation step according to one embodiment to which the present invention can be applied. [Figure 10] FIG. 2 is a diagram showing the order of interpolation steps according to one embodiment to which the present invention can be applied. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The video signal processing method and apparatus according to the present invention can determine an intra prediction mode of a current block, determine reference samples to be used for intra prediction of the current block, determine a predetermined matrix based on the intra prediction mode, and predict the current block based on the reference samples and the matrix.

[0020] In the video signal processing method and apparatus according to the present invention, the step of determining the reference sample may include the steps of determining a neighboring region of the current block, and performing downsampling on the determined neighboring region.

[0021] In the video signal processing method and apparatus according to the present invention, the adjacent region is divided into a plurality of sample groups, each of the sample groups consisting of one or more samples, and a representative value of the sample group is determined by the reference sample, and the representative value may be any one of an average value, a minimum value, a maximum value, a mode value or a median value.

[0022] In the video signal processing method and apparatus according to the present invention, the matrix may be determined by further taking into account coding information of the current block, and the coding information may include the size, shape, angle or direction of an intra prediction mode of the current block.

[0023] In the video signal processing method and apparatus according to the present invention, predicting the current block may include generating a predicted block by applying the matrix to the reference samples.

[0024] In the video signal processing method and apparatus according to the present invention, the step of predicting the current block may further include rearranging all or a part of prediction samples of the generated prediction block.

[0025] In the video signal processing method and apparatus according to the present invention, the step of predicting a current block may further include performing interpolation on the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block.

[0026] In the present invention, various modifications can be implemented and various embodiments can be provided, and specific examples are shown in the drawings and described in detail. However, these specific examples do not limit the present invention to the specific embodiments, and include all modifications, equivalents, or alternatives within the spirit and technical scope of the present invention. When describing each drawing, similar reference numerals are used for similar components.

[0027] Terms such as first, second, etc. may be used to describe an element, but the element is not limited to the terms. The terms are used merely to distinguish one element from another element. For example, a first element may be named a second element, and similarly, the second element may be named a first element, without departing from the scope of the invention. The term "and / or" refers to the inclusion of a combination of related items listed or any one of related items listed.

[0028] When describing an element as being "connected" or "coupled" to another element, it should be understood that it may be directly connected or coupled to the other element, or that there may be other elements between the element and the other element. Conversely, when describing an element as being "directly connected" or "directly coupled" to another element, it should be understood that there are no other elements between the element and the other element.

[0029] The terms used in this application are merely for describing specific embodiments and are not intended to limit the present invention. Unless otherwise clearly indicated in the specification, singular expressions include plural expressions. In this application, terms such as "include" or "have" specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] Unless otherwise defined, all terms, including technical and scientific terms, used herein have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be understood to have meanings consistent with their meanings in the relevant technical literature, and should not be interpreted as ideal or formal meanings unless expressly defined herein.

[0031] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following, the same components in the drawings are designated by the same reference numerals, and duplicated description of the same components will be omitted.

[0032] FIG. 1 is a schematic block diagram showing an encoding device according to an embodiment of the present invention.

[0033] Referring to FIG. 1, the encoding device 100 may include an image division unit 110, prediction units 120, 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filtering unit 150, and a memory 155.

[0034] Each component shown in FIG. 1 is shown alone, and may represent different characteristic functions in the image encoding device, and each component may be configured as a separate piece of hardware. However, for convenience of explanation, each component is described and included as a separate component. At least two of the components may constitute one component, or one component may be divided into multiple components to perform a function. Such comprehensive and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0035] In addition, some components may not be essential components for performing basic functions in the present invention, but may be only selectable components for improving performance. The present invention may be realized by including only components. The components are essential for achieving the essence of the present invention other than the components for improving performance. A structure including only essential components other than the selectable components for improving normality is also included within the scope of the present invention.

[0036] The image division unit 110 can divide the input image into at least one block. In this case, the block may represent a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The division can be performed by at least one of a quad tree, a binary tree, and a ternary tree. A quad tree is a method of dividing an upper layer block into four lower layer blocks, each of which has a width and height half that of the upper layer block. A binary tree is a method of dividing the layer block into two lower layer blocks, each of which has a width or height at least half that of the upper layer block. In the division of the binary tree, the division based on the bifurcations, which has a height half that of the upper layer block, allows the blocks to have not only a square shape but also a non-square shape.

[0037] Hereinafter, in the embodiments of the present invention, the encoding unit may be used as a unit that performs encoding, or may be used as a unit that performs decoding.

[0038] The prediction units 120 and 125 may include an inter prediction unit 120 for performing inter prediction and an intra prediction unit 125 for performing intra prediction. For a prediction unit, it is possible to determine whether to use frame prediction or intra prediction, and to determine specific information (e.g., intra prediction mode, motion vector, reference image, etc.) according to each prediction method. In this case, the processing unit for performing prediction may be different from the processing unit for determining the prediction method and specific content. For example, the prediction method and prediction mode may be determined by the prediction unit, and the prediction may be performed by the transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit 130. In addition, the entropy coding unit 165 may code prediction mode information, motion vector information, etc. used for prediction together with the residual value and transmit them to the decoding device. When one coding mode is used, it is not necessary to generate a prediction block by the prediction units 120 and 125, and the original block may be coded and transmitted to the decoding unit in the same manner.

[0039] The inter prediction unit 120 may predict a prediction unit based on information of at least one image of an image immediately before or immediately after the current image. In some cases, the prediction unit may be predicted based on information of some areas in the current block that have been completely coded. The prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0040] In the reference image interpolation unit, reference image information is provided by the memory 155. Also, pixel information of integer pixels or less can be generated in the reference image. For luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 4 pixel units. For color difference signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information of integer pixels or less in 1 / 8 pixel units.

[0041] The motion prediction unit can perform motion prediction based on the reference image interpolated by the reference image interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), NTS (New Three-Step Search Algorithm) can all be used as a method for calculating a motion vector. According to the interpolated pixel, the motion vector may have a motion vector value in 1 / 2 or 1 / 4 pixel units. The motion prediction unit can predict the current prediction unit by different motion prediction methods. Various methods such as a Skip method, a Merge method, and an Advanced Motion Vector Prediction (AMVP) method can be used as a motion prediction method.

[0042] The intra prediction unit 125 may generate a prediction unit based on reference pixel information adjacent to the current block. Here, the reference pixel information adjacent to the current block is pixel information in the current image. Since the adjacent (surrounding) block of the current prediction unit is an inter-predicted block, when the reference pixel is an inter-predicted pixel, the reference pixel information included in the inter-predicted block may be replaced with reference pixel information of the adjacent intra-predicted block. In other words, when the reference pixel is unavailable, the unavailable reference pixel information may be replaced with at least one of the available reference pixels.

[0043] The prediction mode in intra prediction may include a directional prediction mode that uses reference pixel information based on the prediction direction, and a non-directional mode that does not use directional information during prediction. The mode for predicting the luminance component may be different from the mode for predicting the chrominance component. In addition, the chrominance component may be predicted using the intra prediction mode used to predict the luminance component or the predicted / reconstructed luminance component.

[0044] The intra prediction method may generate a prediction block after applying an Adaptive Intra Smoothing (AIS) filter to reference pixels based on the intra prediction mode. The type of the AIS filter applied to the reference pixels may be different. To perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode located in the prediction unit adjacent to the current prediction block. When predicting the prediction mode of the current prediction unit using mode information predicted from the adjacent prediction unit, if the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the adjacent prediction unit, information indicating that the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the adjacent prediction unit may be transmitted using predetermined flag information. If the intra prediction mode of the current prediction unit is different from the intra prediction mode of the adjacent prediction unit, entropy coding may be performed to encode intra prediction mode information of the current block.

[0045] In addition, the prediction units 120 and 125 may generate residual value information including a difference value between the generated prediction unit and the original block. The generated residual block may be input to the conversion unit .

[0046] The residual block including the residual data may be transformed by the transform unit 130 using a transform of a type such as DCT, DST, etc. In this case, the transform type may be determined based on the intra-prediction mode of the prediction unit used to generate the residual block.

[0047] The quantization unit 135 may quantize the values ​​transformed into the frequency domain by the transformation unit 130. The quantization coefficient may vary depending on the importance of a block or an image. The values ​​calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0048] The rearrangement unit 160 may rearrange coefficient values ​​for the quantized residual block. The rearrangement unit 160 may change coefficients of a two-dimensional block shape into a one-dimensional vector shape using a coefficient scanning method. For example, the rearrangement unit 160 may scan from a DC coefficient to a high frequency region coefficient using a predetermined scanning type and change the coefficients into a one-dimensional vector shape.

[0049] The entropy coding unit 165 can perform entropy coding based on the value calculated by the rearrangement unit 160. The entropy coding can use various coding methods such as Exponential Golomb coding, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0050] The entropy coding unit 165 can perform coding on various information from the rearrangement unit 160 and the prediction units 120 and 125, such as, for example, residual coefficient information and block type information of the coding unit, prediction mode information, division unit information, prediction unit information and transmission unit information, motion vector information, reference image information, block interpolation information, filtering information, etc.

[0051] The entropy coding unit 165 may perform entropy coding on the coefficient values ​​of the coding units input to the rearrangement unit arrangement unit 160 .

[0052] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the values ​​quantized by the quantization unit 135, and perform inverse transform on the values ​​transformed by the transform unit 130. Residual values ​​generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with prediction units predicted by the motion estimation unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0053] The filtering unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0054] The deblocking filter can remove block artifacts caused by boundaries between blocks in the reconstructed image. To determine whether to perform deblocking, it is determined whether to apply the deblocking filter to the current block based on pixels contained in some columns or rows contained in the block. When applying the deblocking filter to a block, a strong filter or a weak filter can be applied based on the required deblocking filtering strength. In addition, when applying the deblocking filter, when performing vertical filtering and horizontal filtering, horizontal filtering and vertical filtering can be processed in parallel.

[0055] The offset correction unit can correct the offset of the deblocked image with respect to the original image on a pixel-by-pixel basis. In order to perform offset correction on a specific image, a method can be used in which the pixels included in the image are divided into a certain number of regions, and then the region to be offset is determined, and the offset is applied to the region, or a method can be used in which the offset is applied by considering edge information of each pixel.

[0056] Adaptive Loop Filtering (ALF) can be performed based on a comparison value between the filtered reconstructed image and the original image. After dividing the pixels included in the image into predetermined groups, one filter that can be applied to the combination is determined. Also, filtering can be performed differently for each group. Regarding information on whether ALF can be applied, each coding unit (CU) can transmit a luminance signal. Also, the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. It is also possible to apply an ALF filter of the same shape (fixed shape) regardless of the characteristics of the block to which it is applied.

[0057] The memory 155 may store the reconstructed block or image calculated by the filtering unit 150. The stored reconstructed block or image may be provided to the prediction units 120 and 125 when performing inter prediction.

[0058] FIG. 2 is a schematic block diagram showing a decoding device according to an embodiment of the present invention.

[0059] Referring to FIG. 2, the decoding device 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filtering unit 240, and a memory 245.

[0060] Each component shown in FIG. 2 is shown alone, and may represent different characteristic functions in the image encoding device, and each component may be configured as a separate piece of hardware. However, for convenience of explanation, each component is described and included as a separate component. At least two of the components may constitute one component, or one component may be divided into multiple components to perform a function. Such comprehensive and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0061] The entropy decoding unit 210 can perform entropy decoding on the input code stream. For example, various methods such as Exponential Golomb coding, CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), etc. can be applied to perform entropy decoding.

[0062] The entropy decoding unit 210 can perform decoding on information related to intra prediction and inter prediction performed by the encoding device.

[0063] The rearrangement unit 215 can rearrange the code stream entropy decoded by the entropy decoding unit 210. The rearrangement unit 215 can reconstruct and rearrange coefficients expressed in one-dimensional vector format to become coefficients in two-dimensional block format. The rearrangement unit 215 can receive information related to the coefficient scan performed by the encoding device and perform rearrangement by a method of performing a reverse scan according to the order of the scan performed by the encoding device.

[0064] The inverse quantization unit 220 may perform inverse quantization based on the quantization parameter and the coefficient values ​​of the rearranged block.

[0065] The inverse transform unit 225 can perform inverse transform on the dequantized transform coefficients with a predetermined transform type. In this case, the transform type can be determined based on at least one of information related to the prediction mode (inter / intra prediction), block size / shape, intra prediction mode, component type (luminance / chrominance component), and partition type (QT, BT, TT, etc.).

[0066] The prediction units 230, 235 generate related information based on the prediction block provided by the entropy decoding unit 210 and can generate the prediction block based on this previously decoded block or image information provided by the memory 245.

[0067] The prediction unit 230, 235 may include a prediction unit discrimination unit, an inter prediction unit, and an intra prediction unit. The prediction unit discrimination unit receives various information such as prediction unit information input by the entropy decoding unit 210, intra prediction mode related information of the intra prediction method, and motion prediction related information of the inter prediction method, classifies prediction units in a current coding unit (CU), and determines whether the prediction unit performs inter prediction or intra prediction. The inter prediction unit 230 may perform inter prediction for the current prediction unit based on information included in at least one image of an image immediately before or immediately after the current image including the current prediction unit, using information necessary for inter prediction of the current prediction unit provided by the encoding device. Alternatively, the inter prediction may be performed based on information of several reconstructed regions in the current image including the current prediction unit. Thus, the several reconstructed regions may be added to a reference image list.

[0068] In order to perform inter prediction, it is possible to determine, based on a coding unit, whether the motion prediction method of the prediction unit included in the coding unit is skip mode, merge mode, AMVP mode, or current image reference mode.

[0069] The intra prediction unit 235 can generate a prediction block based on pixel information in the current image. When the prediction unit is a prediction unit that has performed intra prediction, the prediction unit can perform intra prediction based on intra prediction mode information of the prediction unit provided by the encoding device. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a member that performs filtering on the reference pixels of the current block, and can determine whether to apply a filter based on the prediction mode of the current prediction unit. The AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the encoding device. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter does not need to be applied.

[0070] When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the value of the pixel that is to be interpolated for the reference pixel, the reference pixel interpolation unit can perform interpolation on the reference pixel to generate reference pixels in pixel units of integer values ​​or less. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without performing interpolation on the reference pixel, the reference pixel does not need to be interpolated. When the prediction mode of the current block is a DC mode, the DC filter can generate a prediction block by filtering.

[0071] The reconstructed block or image may be provided to a filtering unit 240, which may comprise a deblocking filter, an offset correction unit, and an ALF.

[0072] From the encoding device, information regarding whether a deblocking filter is applied to the block or image may be received, and if a deblocking filter is applied, information regarding whether a strong filter or a weak filter is applied may be received. The deblocking filter of the decoding device may receive the deblocking filter related information provided by the encoding device. The decoding device may perform deblocking filtering on the block.

[0073] When encoding, the offset correction unit can perform offset correction on the reconstructed image based on the offset correction type and offset value information applied to the image.

[0074] The ALF may be applied to a coding unit based on encoder-provided information regarding whether to apply the ALF, ALF coefficient information, etc. Such ALF information may be provided by being included in a particular parameter set.

[0075] The memory 245 can store the reconstructed image or block as a reference image or block, and can provide the reconstructed image to an output.

[0076] FIG. 3 is a diagram showing block division types according to an embodiment to which the present invention can be applied.

[0077] Referring to FIG. 3, blocks a to s can be obtained based on the division setting and division method, and additional block shapes (not shown) can also be obtained.

[0078] As an example (1), asymmetric splitting may be allowed for tree-based splitting. For example, for a binary tree, the blocks may be b, c, or b to g. If a flag indicating that asymmetric splitting is allowed is not activated explicitly or implicitly based on the encoding / decoding setting, the available candidate blocks may be b or c. If a flag indicating that asymmetric splitting is allowed is activated, the available candidate blocks may be b, d, e (horizontal splitting in this example), or c, f, g (vertical splitting in this example).

[0079] In the above example, the length ratio of the left:right or top:bottom of the asymmetric division is described as 1:3 or 3:1, but this is not limited to this, and there may be candidate groups with other ratios (e.g., 1:2, 1:4, 2:3, 2:5, 3:5, etc.) depending on the encoding settings.

[0080] Various examples of partition information generated in binary tree partitioning (1:1, 1:3, and 3:1 candidate groups in this example) will be shown below.

[0081] For example, in addition to a flag indicating whether or not to perform a division and a flag indicating the division direction, a flag indicating the division type can also be generated. In this case, the division type may represent symmetric or asymmetric division. Here, if asymmetric division is determined to be the division type, a flag indicating the division ratio can be generated. Also, an index can be assigned based on a predetermined candidate group. If a 1:3 or 3:1 division ratio is supported as a candidate group, the division ratio can be selected by a 1-bit flag.

[0082] Alternatively, in addition to a flag indicating whether or not to perform division and a flag indicating the division direction, a flag indicating the division ratio can be generated. In this example, the candidate group for the division ratio may include candidates having a symmetric ratio of 1:1.

[0083] In the present invention, we assume that the binary tree partition has a structure as in the above example (if the flag indicating that asymmetric partitioning is allowed is activated). Unless otherwise stated, the binary tree represents a symmetric binary tree.

[0084] As an example (2), tree-based partitioning can allow additional tree partitioning. For example, partitioning can be performed on a ternary tree, a quad type tree, an octa tree, etc., which can obtain n partitioned blocks (3, 4, 8, n being an integer in this example). For a ternary tree, the supported blocks (when partitioned into multiple blocks in this example) may be h to m. For a quad tree, the supported blocks may be n to p. For an octa tree, the supported blocks may be q. Whether to support the tree-based partitioning can be implicitly determined or related information can be explicitly generated based on the encoding / decoding configuration. Note that it may be used alone or mixed with a binary tree, a quad tree partitioning, etc. based on the encoding / decoding configuration.

[0085] For example, for a binary tree, blocks such as b and c can be used. When a binary tree and a ternary tree are mixed (in this example, it is assumed that the range of use of the binary tree and the range of use of the ternary tree overlap partially), blocks such as b, c, i, and l can also be used. If a flag indicating that additional divisions other than the current tree are allowed is explicitly or implicitly deactivated based on the encoding / decoding setting, the obtainable candidate block may be b or c. If activated, the obtainable candidate block may be b, i or b, h, i, j (horizontal division in this example), or may be c, l or c, k, l, m (vertical division in this example).

[0086] In the above example, the length ratio of the left:middle:right or top:middle:bottom of the ternary tree division is assumed to be 2:1:1, 1:2:1 or 1:1:2, but this is not limited to this and other ratios can also be set based on the encoding settings.

[0087] Below, an example of division information generated by ternary tree division (1:2:1 candidate in this example) is shown.

[0088] For example, in addition to a flag indicating whether or not a division is to be performed and a flag indicating the division direction, a flag indicating the division type may be generated, in which case the division type may represent a binary tree or ternary tree division.

[0089] In the present invention, adaptive encoding / decoding settings can be applied based on the partitioning scheme.

[0090] As an example, the partitioning scheme can be determined based on the type of block, e.g., coding blocks and transform blocks can use a quadtree partitioning scheme, and prediction blocks can use a quadtree and binary tree (or ternary tree, etc.) partitioning scheme.

[0091] As an example, the division method can be determined based on the size of the block. For example, a quad-tree division method can be used for some ranges between the maximum and minimum values ​​of the block (e.g., a×b to c×d, where the latter is larger). A binary tree (or ternary tree, etc.) division can be performed for some ranges (e.g., e×f to g×h). In this case, range information according to the division method can be explicitly generated or implicitly determined. It can also be used when the ranges overlap.

[0092] As an example, the division method can be determined based on the shape of the block (or the block before division). For example, if the shape of the block is a square, quadtree or binary tree (or ternary tree, etc.) division can be performed. Or, if the shape of the block is a rectangle, division based on a binary tree (or ternary tree, etc.) can be performed.

[0093] As an example, the partitioning setting may be determined based on the type of block. For example, in a tree-based partitioning, the coding block and the prediction block may use quadtree partitioning, and the transformation block may use binary tree partitioning. Or, the partitioning depth allowed for the coding block may be set as m. The partitioning depth allowed for the prediction block may be set as n, and the partitioning depth allowed for the transformation block may be set as o. m, n, and o may be the same or different.

[0094] As an example, the partitioning settings can be determined based on the size of the block. For example, quad-tree partitioning can be used in some ranges of the block (e.g., a×b to c×d), binary-tree partitioning can be used in some ranges (e.g., e×f to g×h. In this example, c×d is larger than g×h), and ternary-tree partitioning can be used in some ranges (e.g., i×j to k×l. In this example, g×h is equal to or larger than k×l). In this case, the ranges may include all ranges between the maximum and minimum values ​​of the block. The ranges may have non-overlapping settings or overlapping settings. For example, the minimum values ​​of some ranges may be the same as the maximum values ​​of some ranges, or the minimum values ​​of some ranges may be smaller than the maximum values ​​of some ranges. In the case of overlapping ranges, the partitioning scheme with the larger maximum value may have priority or information regarding which partitioning scheme to use may be explicitly generated. In other words, in a division method having a priority, it is possible to determine whether to use a division method with a lower priority based on the division result, or it is possible to determine which division method to use based on division method selection information.

[0095] As an example, the partitioning settings can be determined based on the shape of the block: for example, if the shape of the block is square, then quadtree partitioning can be used, or if the shape of the block is rectangular, then binary or ternary tree partitioning can be used.

[0096] As an example, a partition setting can be determined based on encoding / decoding information (e.g., slice type, color component, encoding mode, etc.). For example, when the slice type is I, quadtree (or binary tree, ternary tree) partitioning can be used within some ranges (e.g., a×b to c×d), when the slice type is P, it can be used within some ranges (e.g., e×f to g×h), and when the slice type is B, it can be used within some ranges (e.g., i×j to k×l). Note that when the slice type is I, the partitioning allowable depth of the quadtree (or binary tree, ternary tree partitioning) partitioning may be set to m, when the slice type is P, the partitioning allowable depth may be set to n, and when the slice type is B, the partitioning allowable depth may be set to o. m, n, and o may be the same or different. Some slice types may have similar settings to other slices (e.g., slices P and B).

[0097] As another example, when the color component is a luminance component, the allowable depth of the quadtree (or binary tree, ternary tree) division may be set to m, and when the color component is a chrominance component, it may be set to n, where m and n may be the same or different. Note that when the color component is a luminance component, the range of the quadtree (or binary tree, ternary tree) division (e.g., a×b to c×d) may be the same as or different from the range of the quadtree (or binary tree, ternary tree) division (e.g., e×f to g×h) when the color component is a chrominance component.

[0098] As another example, when the coding mode is Intra, the allowable depth of quadtree (or binary tree, ternary tree) division may be m, and when the coding mode is Inter, it may be n (in this example, n is assumed to be greater than m), where m and n may be the same or different. Note that the range of quadtree (or binary tree, ternary tree) division when the coding mode is Intra may be the same as or different from the range of quadtree (or binary tree, ternary tree) division when the coding mode is Inter.

[0099] For the above example, information regarding whether to support adaptive partition candidate set structure according to encoding / decoding information can be explicitly generated or implicitly determined.

[0100] The above example describes the determination of the partitioning method and partitioning setting based on the encoding / decoding setting. The above example shows some situations depending on each element, but there may be other variations. In addition, the partitioning method and partitioning setting may be determined based on a combination of multiple elements. For example, the partitioning method and partitioning setting may be determined based on the type, size, shape, encoding / decoding information, etc. of the block.

[0101] In addition, it is possible to determine whether or not to allow application situations such as the above example by implicitly determining elements related to the division method, settings, and the like in the above example, or by explicitly generating information.

[0102] The division depth in the division setting indicates the number of times to perform spatial division based on the initial block (the division depth of the initial block in this example is 0). The larger the division depth, the smaller the blocks that can be divided. This means that the depth-related setting can be set differently based on the division method. For example, in a tree-based division method, the division depth of a binary tree can share the same depth as that of a ternary tree. The division depth of a quad tree can be a different depth from that of a binary tree. Each depth can be used based on the type of tree.

[0103] In the above example, when using each division depth based on the type of tree, the division depth at the division start position of the tree (the block before division in this example) can be set to 0. The division depth can be calculated centered on the division start position, without being based on the division range of each tree (the maximum value in this example).

[0104] FIG. 4 is a diagram showing a block division method based on a tree structure according to an embodiment to which the present invention can be applied.

[0105] In the drawings, the thick solid lines represent basic coding blocks, the thick dotted lines represent quadtree partitioning limits, the double solid lines represent symmetric binary tree partitioning limits, the solid lines represent ternary tree partitioning limits, and the thin solid lines represent asymmetric binary tree partitioning limits. All lines other than the thick solid lines represent the limits of partitioning by each partitioning method. The partitioning settings described below (e.g., partitioning type, partitioning information, partitioning information arrangement order, etc.) are not limited to those in the example, and may have various modified examples.

[0106] For ease of explanation, it is assumed that the upper left, upper right, lower left, and lower right blocks (N×N, 64×64) based on the basic coding block (2N×2N, 128×128) have their own block division settings. First, four sub-blocks are obtained by one division operation (division depth 0->1, i.e., the division depth increases by 1) on the initial block, and it is also assumed that in the quadtree division setting, the maximum coding block is 128×128, the minimum coding block is 8×8, and the maximum division depth is 4. This is a common setting that is applied to each block.

[0107] (1st. Top left block, A1~A6) In this example, assuming that a single tree type division (quadtree in this example) is supported, the size and shape of obtainable blocks can be determined by one block division setting such as the maximum coding block, the minimum coding block, the division depth, etc. In this example, assuming that there is one block obtainable by division (division into two in the horizontal and vertical directions), the division information required for one division operation (with the block before division, 4M×4N, as a reference, and the division depth increasing by 1) is a flag indicating whether or not division is to be performed (in this example, if 0, division is not performed, and if 1, division is performed), and obtainable candidates may be 4M×4N and 2M×2N.

[0108] (2nd upper right block, A7~A11) In this example, multiple tree-based partitioning (quad-tree, binary tree in this example) is supported, and multiple block partitioning settings can determine the size and shape of the blocks that can be obtained. In this example, it is assumed that the maximum coding block for the binary tree is 64x64, the minimum coding block is 4 in length, and the maximum partitioning depth is 4.

[0109] In this example, assuming that the number of blocks that can be obtained by splitting is two (two or four in this example), the splitting information required for one splitting operation (the quadtree splitting depth increases by 1) is a flag indicating whether to perform splitting, a flag indicating the split type, a flag indicating the split type, and a flag indicating the split direction, and the obtainable candidates may be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N.

[0110] If the quadtree division range and binary tree division range overlap (i.e., the range in which quadtree division and binary tree division can be performed in the current step) and the current block (state before division) is a block obtained by quadtree division (a block obtained by quadtree division in a parent block <when ​​the division depth is 1 less than the current one>), the division information can be arranged by classifying as follows. In other words, if a block supported based on each division setting can be obtained by multiple division methods, the division information can be generated by classifying as follows.

[0111] (1) When quadtree division and binary tree division overlap [Table 1]

[0112] In the above table, a is a flag indicating whether or not quadtree division is performed, and 1 indicates that quadtree division (QT) is performed. If the flag is 0, a flag b indicating whether or not binary tree division is performed is checked. If b is 0, no division is performed in the block (No Split). If b is 1, binary tree division is performed.

[0113] c is a flag indicating the division direction. If c is 0, horizontal division (hor) is performed. If c is 1, vertical division (ver) is performed. d is a flag indicating the division type. If d is 0, symmetric division (SBT: Symmetric Binary Tree) is performed. If d is 1, asymmetric division (ABT: Asymmetric Binary Tree) is performed. Only when d is 1, is information related to the detailed division ratio in asymmetric division (1 / 4 or 3 / 4) checked. If d is 0, in the left / right blocks or top / bottom blocks, the ratio between the left block and the top block is 1 / 4, and the ratio between the right block and the bottom block is 3 / 4. If d is 1, it is reversed.

[0114] (2) If only binary tree partitioning is feasible, In the above table, the division information can be expressed by flags b through e other than a.

[0115] As for block A7 in FIG. 4, quadtree partitioning can be performed on the blocks (A7 to A11) before partitioning (i.e., quadtree partitioning can be performed, but binary tree partitioning was performed without quadtree partitioning), so this belongs to the case where partition information is generated in (1).

[0116] On the other hand, for A8 to A11, the blocks before division (A8 to A11) were not divided into quadtrees but into binary trees (i.e., the blocks<A8~A11> In the case where further quadtree division cannot be performed in (2), this belongs to the case where division information is generated in (2).

[0117] (3rd, bottom left block, A12~A15) In this example, multiple tree-based partitioning (quad-tree, binary-tree, and ternary-tree in this example) is supported, and the size and shape of the obtainable blocks can be determined by multiple block partitioning settings. In this example, it is assumed that the maximum coding block for binary / ternary trees is 64x64, the minimum coding block is 4 in length, and the maximum partitioning depth is 4.

[0118] In this example, assuming that there are two or more blocks that can be obtained by splitting (2, 3, or 4 in this example), the splitting information required for one splitting operation is a flag indicating whether to perform splitting, a flag indicating the split type, and a flag indicating the splitting direction, and the obtainable candidates may be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×2N / 4M×N, M×4N / 2M×4N / M×4N.

[0119] If the quadtree division range and the binary / ternary tree division range overlap, and the current block is a block obtained by quadtree division, the division information can be arranged by dividing it as follows:

[0120] (1) When quadtree division and binary / ternary tree division overlap [Table 2]

[0121] In the table, a is a flag indicating whether or not quadtree division is performed, and when it is 1, it indicates that quadtree division is performed. When the flag is 0, a flag b indicating whether or not binary or ternary tree division is performed is checked. When b is 0, no division is performed in that block. When b is 1, binary or ternary tree division is performed.

[0122] c is a flag indicating the split direction. If c is 0, a horizontal split is performed. If c is 1, a vertical split is performed. d is a flag indicating the split category. If d is 0, a binary tree split (BT) is performed. If d is 1, a ternary tree split (TT) is performed.

[0123] (2) When only binary / ternary tree division is feasible In the above table, division information can be expressed by flags b through d other than a.

[0124] Regarding blocks A12 and A15 in FIG. 4, quadtree partitioning can be performed on the blocks (A12 to A15) before partitioning, so this belongs to the case of generating partition information in (1).

[0125] Conversely, for A13 and A14, ternary tree division was performed without quad tree division in the blocks (A13, A14) before division, so this belongs to the case of generating division information in (2).

[0126] (4th, bottom left block, A16~A20) In this example, multiple tree-based partitioning (quad-tree, binary-tree, and ternary-tree in this example) is supported, and the size and shape of the obtainable blocks can be determined by multiple block partitioning settings. In this example, it is assumed that the maximum coding block for binary / ternary trees is 64x64, the minimum coding block is 4 in length, and the maximum partitioning depth is 4.

[0127] In this example, assuming that there are two or more blocks that can be obtained by splitting (2, 3, or 4 in this example), the splitting information required for one splitting operation is a flag indicating whether to perform splitting, a flag indicating the split category, a flag indicating the split type, and a flag indicating the split direction, and the obtainable candidates may be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N, 4M×N / 4M×2N / 4M×N, M×4N / 2M×4N / M×4N.

[0128] If the quadtree division range and the binary / ternary tree division range overlap, and the current block is a block obtained by quadtree division, the division information can be arranged by dividing it as follows:

[0129] (1) When quadtree division and binary / ternary tree division overlap [Table 3]

[0130] In the table, a is a flag indicating whether or not to perform quadtree division, and if it is 1, then quadtree division is performed. If the flag is 0, then b is checked, which is a flag indicating whether or not to perform binary tree division. If b is 0, then no further division is performed on that block. If b is 1, then binary or ternary tree division is performed.

[0131] c is a flag indicating the split direction. If c is 0, horizontal split is performed. If c is 1, vertical split is performed. d is a flag indicating the split category. If d is 0, ternary tree split is performed. If d is 1, binary tree split is performed. If d is 1, e, which is a flag indicating the split type, is checked, and if e is 0, symmetric split is performed, and if e is 1, asymmetric split is performed. If e is 1, information on the detailed split ratio for asymmetric split is checked. This is the same as the previous example.

[0132] (2) When only binary / ternary tree division is feasible In the above table, the division information can be expressed by flags b to f other than a.

[0133] Block A20 in FIG. 4 belongs to the case of generating division information in (1) since quadtree division can be performed on the blocks (A16 to A19) before division.

[0134] Conversely, for A16 to A19, quad-tree division was not performed on the blocks before division (A16 to A19), but binary-tree division was performed, so this falls under the case of generating division information in (2).

[0135] FIG. 5 is a diagram illustrating a process of performing intra prediction on a current block based on a matrix according to an embodiment of the present invention.

[0136] Referring to FIG. 5, an intra prediction mode to be used for intra prediction of a current block may be determined (S500).

[0137] When performing intra prediction of a current block, the encoding / decoding apparatus may determine an intra prediction mode. The current block may be a coding block (CU), a prediction block (PU), a transform block (TU), or any one of these blocks.

[0138] (Example 1) The intra prediction mode may be determined based on information conveying a signal. The information may specify any one of N intra prediction modes predefined in the encoding / decoding device. The predefined intra prediction modes represent all intra prediction modes available for the current block. N may be a natural number (e.g., 67, 35, 11) that is less than 67 and greater than or equal to 11. The value of N may be determined based on the size of the current block. For example, if the current block is less than 8×8, N is determined to be 35. Otherwise, N may be determined to be any one of 19 or 11.

[0139] (Example 2) The intra prediction mode may be determined by a default mode or index predefined in the encoding / decoding device. The default mode may be at least one of a planar mode (index 0), a DC mode (index 1), a horizontal mode (index 18), a vertical mode (index 50), and a diagonal mode (indexes 2, 34, and 66). Here, the index corresponds to the case where there are 67 predefined intra prediction modes, and a different index may be assigned for each mode based on the value of N.

[0140] (Example 3) The intra prediction mode can be variably determined based on coding information. Here, the coding information may include not only information conveying a signal coded in a coding device, but also information derived based on information conveying a signal in a decoding device. The coding information may be information related to at least one of a current block or a neighboring block. The neighboring block may include a spatial and / or temporal neighboring block of the current block, and the spatial neighboring block may represent a block adjacent to at least one of the left side, upper layer, upper left layer, lower left layer, or upper right layer of the current block.

[0141] The coding information may include a size / shape of a block, availability of a block, a division type, a division count, a component type, a prediction mode, information related to an intra prediction mode, an inter mode, motion information, a transform type, a transform skip mode, information related to a non-zero residual coefficient, a scan order, a color format, loop filter information, and the like. The size of the block may be represented by any one of a width or a height, a minimum value / maximum value in the width and height, a sum of the width and the height, a number of samples belonging to the block, and the like. The availability of the block may be determined by considering a position of the block, a range of a parallel processing area, a decoding order, and the like. The prediction mode may represent information for indicating an intra mode or an inter mode. The information related to the intra prediction mode may include information related to whether the intra prediction mode is a non-directional mode, whether the intra prediction mode is a vertical / horizontal mode, a directionality of the intra prediction mode, a number of intra prediction modes predefined in the encoding / decoding device, and the like. The inter mode may represent information for indicating a merge / skip mode, an AMVP mode, or a current image reference mode. The current image reference mode represents a method of predicting a current block using a reconstructed area of ​​a current image. The current image may be an image to which the current block belongs. The current image may be added to a reference image list to perform inter prediction. The current image may be arranged after a short-term reference image or a long-term reference image in the reference image list. The motion information may include a prediction direction flag, a motion vector, a reference image index, etc.

[0142] (Example 4) The intra prediction mode can also be derived based on an MPM list and an MPM index. The MPM list includes multiple MPMs. The MPM can be determined based on the intra prediction modes of spatial / temporal neighboring blocks of the current block. The number of MPMs is x, where x can be 3, 4, 5, 6, or a larger integer.

[0143] For example, the MPM list may include at least one of intra prediction modes mode A, (mode An), (mode A+n) or a default mode of the neighboring blocks. The value of n may be 1, 2, 3, 4 or a larger integer. The neighboring blocks may represent neighboring blocks on the left and / or upper layer of the current block. The default mode may be at least one of a Planar mode, a DC mode, or a predefined directional mode. The predefined directional mode may include at least one of a horizontal mode (mode V), a vertical mode (mode H), (mode Vk), (mode V+k), (mode Hk) or (mode H+k).

[0144] The MPM index may specify an MPM that is the same as the intra prediction mode of the current block among the MPMs in the MPM list, that is, the MPM specified by the MPM index may be set as the intra prediction mode of the current block.

[0145] The intra prediction mode of the current block may be determined by selectively using any one of the first to fourth embodiments. Also, the intra prediction mode of the current block may be determined by a combination of at least two of the first to fourth embodiments. A predetermined flag may be used for the selection. In this case, the flag may be coded by a coding device and transmitted as a signal.

[0146] Referring to FIG. 5, a reference sample to be used for intra prediction of a current block may be determined (S510).

[0147] The reference samples may be derived from neighboring regions (surrounding regions) of the current block, which may include at least one of the left, right, top, bottom left, top left, bottom right, or top right layers of the current block.

[0148] The adjacent region may include one or more sample lines. Specifically, the number of sample lines belonging to the adjacent region is k, where k may be 1, 2, 3, 4, or a natural number greater than these. The value of k may be a fixed value previously determined in the encoding / decoding device, or may be variably determined based on the encoding information. For example, if the current block is a first size (e.g., 4×4, 4×8, 8×4), the adjacent region may be arranged with one sample line. If the current block is a second size (e.g., 8×8, 16×16, etc.), the adjacent region may be arranged with two sample lines. The sample line may be determined in a vertical or horizontal direction based on the position of the adjacent region. Also, the sample line may contact the current block, or may be separated by a predetermined distance in the vertical and / or horizontal directions based on the current block.

[0149] The plurality of sample lines may be continuous in the vertical and / or horizontal directions with respect to the current block, or may be spaced apart from each other by a predetermined distance. In one embodiment, if there are two sample lines in the upper layer of the current block, the two sample lines are named as the first and second sample lines in an upward direction from the lowest sample line of the two lines. In this case, the first and second sample lines may contact each other or may be spaced apart by a predetermined distance. Here, the predetermined distance may be represented by i line lengths (i.e., widths or heights). Here, i may be 0, 1, 2, 3 or a larger natural number. In one embodiment, if there are three sample lines in the upper layer of the current block, the plurality of sample lines are named as the first, second and third sample lines in an upward direction from the lowest sample line of the multiple lines. In this case, the first and second sample lines may contact each other, and the second and third sample lines may contact each other. Alternatively, the first to third sample lines may be spaced apart by the predetermined distance. In this case, the interval (d1) between the first and second sample lines may be the same as the interval (d2) between the second and third sample lines. Alternatively, d1 may be set to be larger than d2. Conversely, d1 may be set to be smaller than d2. As an embodiment, when there are four or more sample lines in the upper layer of the current block, the four sample lines may be determined in the same manner as in the case of the three sample lines. Note that this embodiment is not only applicable to the sample lines located in the upper layer, but also to the sample lines located on the left side. Here, a detailed description is omitted.

[0150] The reference sample can be derived by utilizing all or some of the samples belonging to the adjacent region.

[0151] (Example 1) Some samples in the adjacent region may be samples at positions that are pre-promised in the encoding / decoding device. The pre-promised positions may include at least one of the leftmost sample, the rightmost sample, or the middle sample of the upper layer sample line. The pre-promised positions may include at least one of the topmost sample, the bottommost sample, or the middle sample of the left layer sample line. Or, the pre-promised positions may include at least one of the odd-numbered samples of the upper layer and / or left layer sample line, or at least one of the odd-numbered samples. Or, the pre-promised positions may include samples of the upper layer sample line having an x-coordinate that is a multiple of j, or samples of the left layer sample line having a y-coordinate that is a multiple of j. Here, j may be a natural number that is 2, 3, 4, or greater.

[0152] (Example 2) Some samples of the adjacent region can also be variably determined based on the coding information. Here, the coding information is as described above. Here, detailed description is omitted.

[0153] Several samples can be specified by selectively using any one of the above-mentioned embodiments 1 or 2, or by using a combination of the above-mentioned embodiments 1 and 2. In this case, as described above, the intervals between several samples can be set in the same manner, but are not limited thereto. The intervals between several samples can also be set to be different.

[0154] The number of the some samples may be 1, 2, 3, 4 or more predefined in the encoding / decoding device. The number of the some samples may be defined differently for the left neighboring region and the upper neighboring region of the current block. For example, if the width of the current block is greater than the height, the number of the some samples (numSamA) belonging to the upper neighboring region may be greater than the number of the some samples (numSamL) belonging to the left neighboring region. Conversely, if the width of the current block is less than the height, numSamA may be less than numSamL. Alternatively, the number of the some samples may be variably determined based on the encoding information.

[0155] The samples of the neighboring region may be prediction samples or reconstructed samples. The prediction samples can be obtained by intra prediction or inter prediction. The reconstructed samples may be reconstructed samples before applying a loop filter or reconstructed samples after applying a loop filter.

[0156] In addition, the reference sample may be derived directly from samples of the neighboring region (CASE 1), or may be derived by performing downsampling on samples of the neighboring region (CASE 2). Either one of CASE 1 and CASE 2 may be selectively used. The selection may be made based on the coding information. For example, if the size of the current block is less than a predetermined threshold, the reference sample may be derived based on CASE 1. Otherwise, the reference sample may be derived based on CASE 2. Here, the size may be expressed as any one of the width, height, maximum / minimum value of width and height, ratio of width and height, or product of width and height of the current block. As an example, if the current block is less than 8×8, the reference sample may be derived from samples of the neighboring region. Otherwise, the reference sample may be derived by performing downsampling on samples of the neighboring region. The downsampling method will be further described with reference to FIG. 6 and FIG. 7.

[0157] Referring to FIG. 5, matrices to be used for matrix-based intra prediction may be determined (S520).

[0158] The matrix may be determined according to at least one of the intra prediction mode determined in step S500 or the size of the current block. Alternatively, the matrix may be determined by restricting the consideration to only the intra prediction mode of the current block. Alternatively, the matrix may be determined by restricting the consideration to only the size of the current block. The size may be expressed as the minimum / maximum value of the width and height of either the width or the height, the sum of the width and the height, the number of samples belonging to the current block, or the like. However, the present invention is not limited thereto, and the matrix may be determined by further considering coding information related to the current block. Here, the coding information is as described above. Here, detailed description will be omitted.

[0159] Specifically, the matrices pre-specified in the encoding / decoding device may be divided into a plurality of matrix groups. The plurality of matrix groups may be composed of a first matrix group, a second matrix group, ..., an m-th matrix group. Here, m may be 2, 3, 4, 5, or a larger natural number. Based on the size of the current block, the current block may selectively use any one of the plurality of matrix groups. For example, if the size of the current block is 4x4, the first matrix group may be used. If the size of the current block is 8x4, 4x8, or 8x8, the second matrix group may be used. In addition, in other cases, the third matrix group may be used. The matrix group selected based on the size of the current block may include one or more matrix candidates. Any one of the plurality of matrix candidates may be determined according to the matrix of the current block. The determination may be made based on the coding information (e.g., intra prediction mode) of the current block.

[0160] The number of the pre-committed matrices may be equal to the number of the pre-defined intra prediction modes. The number of the pre-committed matrices may be less than the number of the pre-defined intra prediction modes. In this case, one matrix may match with a plurality of intra prediction modes. For example, one matrix may match with two intra prediction modes. In this case, the number of the pre-committed matrices may be a value of 1 / 2 the number of the pre-defined intra prediction modes. However, the number of the pre-committed matrices may be 3, 4, 5, 6 or more.

[0161] In one embodiment, the matching can be determined by considering the directionality and / or symmetry of the intra prediction modes.

[0162] The predefined intra prediction modes may include a directional mode having a predetermined angle. The directional modes may be divided into a first mode group having horizontal directivity and a second mode group having vertical directivity. If the number of directional modes is 65, the first mode group may be set to be modes belonging to indexes 2 to 34, and the second mode group may be set to be modes belonging to indexes 34 to 66.

[0163] The encoding / decoding apparatus defines only matrices for a first mode group. The second mode group may similarly use matrices defined for the first mode group. Conversely, the encoding / decoding apparatus defines only matrices for a second mode group. The first mode group may similarly use matrices defined for the second mode group. In this case, the number of matrices pre-defined may be a value of 1 / 2 times the number of pre-defined intra prediction modes. In one embodiment, when the number of symmetrical mode groups is x, the number of matrices pre-defined may be a value of 1 / x times the number of pre-defined intra prediction modes. Here, x may be 3, 4, or more.

[0164] The symmetry may include symmetry of prediction angles between a mode having vertical directivity and a mode having horizontal directivity with respect to an intra prediction mode having an angle of -45°. Here, the intra prediction mode having a direction has a prediction angle (PredAngle) according to each direction. Here, the mode having vertical directivity may include a mode having an angle of -45°<(PredAngle)-45° along the x-axis direction from the intra prediction mode with the angle of -45° as a reference. Here, the mode having horizontal directivity may include a mode having an angle of -45°<(PredAngle)-45° along the y-axis direction from the mode other than the intra prediction mode with the angle of -45° as a reference.

[0165] Referring to FIG. 5, a current block can be predicted based on the reference samples and the matrix (S530).

[0166] When the reference sample is determined in step S510 and the matrix is ​​determined in step S520, the encoding / decoding apparatus can predict the current block based on the reference sample and the matrix.

[0167] The step of predicting the current block may include applying the matrix to the reference sample to obtain a predicted sample of the DS block (hereinafter referred to as a first predicted sample). The DS block may represent the current block, or may represent a downsampled current block. That is, the DS block may have a size similar to that of the current block. The size of the current block may have a size of 1 / 2, 1 / 4, 1 / 8, or 1 / 16 (at least one of width or height). For example, if the current block is a 4×4, 4×8, or 8×4 block, the DS block may be a 4×4 block. Or, if the current block is an 8×8, 8×16, or 16×8 block, the DS block may be a 4×4 or 8×8 block. Or, if the current block is 16×16 or larger, the DS block may be an 8×8 or 16×16 block. However, the DS block is not limited to a square block, but may be a non-square block. Or, the DS block may be limited to a square block. Here, applying the matrix may include multiplying the reference samples by weight values ​​obtained from the matrix.

[0168] The step of obtaining the first predicted sample may include at least one of the steps of adding an offset value or filtering.

[0169] The step of obtaining the first predicted samples may further include performing rearrangement on the first predicted samples, and the rearrangement may be performed only if one matrix matches multiple intra prediction modes.

[0170] Alternatively, the rearrangement may be performed when the intra prediction mode of the current block belongs to a first mode group having a horizontal directionality. For example, when the intra prediction mode of the current block belongs to a first mode group having a horizontal directionality, the first predicted samples of the DS block are rearranged. When the intra prediction mode of the current block belongs to a second mode group having a vertical directionality, the first predicted samples of the DS block may not be rearranged.

[0171] Conversely, if the intra prediction mode of the current block belongs to a first mode group having a vertical directionality, the rearrangement may be performed. For example, if the intra prediction mode of the current block belongs to a first mode group having a horizontal directionality, the first predicted samples of the DS block are not rearranged. If the intra prediction mode of the current block belongs to a second mode group having a vertical directionality, the first predicted samples of the DS block may be rearranged.

[0172] The rearrangement may be performed as shown in Equation 1 below, where x may represent an x-axis coordinate value and y may represent a y-axis coordinate value. That is, the rearrangement may represent a process of allocating a first predicted sample of (x, y) coordinate to (y, x) coordinate.

[0173] [Formula 1] 1st predicted sample[x][y] = 1st predicted sample[y][x] Alternatively, the reordering of the present invention may refer to a process in which the DS block of first predicted samples is rotated by a predetermined angle, where the predetermined angle may refer to 90 degrees or 180 degrees in a clockwise direction, or may refer to 90 degrees or 180 degrees in a counterclockwise direction.

[0174] The step of predicting the current block may further include a step of upsampling the current block based on at least one of adjacent reconstructed samples or the first predicted sample to obtain a second predicted sample.

[0175] In the upsampling process, whether or not to perform the upsampling or at least one of the methods of performing the upsampling may be determined based on the coding information of the current block. For example, whether or not to perform the upsampling or at least one of the methods of performing the upsampling may be determined based on the size of the DS block consisting of the first predicted samples and the size of the current block. The size of the block may be represented by any one of width or height, minimum / maximum value of width and height, sum of width and height, number of samples belonging to the block, etc.

[0176] Only when the size of the DS block consisting of the first predicted samples is smaller than the size of the current block, it can be determined whether or not to perform the upsampling.

[0177] The method of upsampling may include allocating the first predicted samples to a predetermined position in the current block using a ratio between a size of a DS block consisting of the first predicted samples and a size of the current block, and performing interpolation on a residual area in the block. The residual area may represent an area in the current block other than an area allocated to the first predicted samples. The method of allocating the first predicted samples and interpolating the residual area will be described in detail with reference to Figures 8 to 10.

[0178] FIG. 6 is a diagram showing a method for determining a reference sample by performing downsampling on a neighboring block according to an embodiment to which the present invention can be applied.

[0179] Referring to Fig. 6, (a) in Fig. 6 shows a case where the neighboring regions used for intra prediction are located on the left side and upper layer of the current block. In one embodiment, the sample line located on the left side of the current block contacts the current block and consists of one sample line in the vertical direction. The sample line located on the upper layer of the current block contacts the current block and consists of one sample line in the horizontal direction.

[0180] The reference sample may include a downsampled region formed by downsampling a neighboring region of the current block.

[0181] The downsampling can be determined from the average, maximum, minimum, mode or filtered value of all or some of the samples belonging to the contiguous region.

[0182] In the case of derivation from the average value, the downsampled region can be formed in such a way that the average value of N different samples is assigned to a sample of the downsampled region.

[0183] The N different samples may be consecutively arranged samples or may be spaced apart by a predetermined interval. The predetermined interval is an interval of one or more sample sizes. When there are multiple intervals, the multiple intervals may be uniform or non-uniform. (Here, N is greater than 2 and less than the total number of samples belonging to the adjacent regions.) Note that a combination of the N different samples is called a sample group. In this case, the first sample group and the second sample group may or may not overlap.

[0184] As an example, FIG. 6 shows a case where N is 2, two sample groups do not overlap each other, and the average value of two samples in each sample group is assigned to one sample in the downsampling region for downsampling.

[0185] Alternatively, three consecutive samples (S1, S2, S3) may constitute a first sample group, and the average value of the three samples in the first sample group may be assigned to the sample in the downsampled region (DS1). Three consecutive samples (S2, S3, S4) may constitute a second sample group, and the average value of the three samples in the second sample group may be assigned to the sample in the downsampled region (DS2).

[0186] Or, the minimum or maximum value of the two samples (S1, S2) belonging to the first sample group can be determined and then assigned to the sample (DS1) in the downsampling region. Similarly, the minimum or maximum value of the two samples (S3, S4) belonging to the second sample group can be determined and then assigned to the sample (DS2) in the downsampling region. The same can be used when the first / second sample group is composed of three samples.

[0187] Alternatively, in the upper adjacent region, a sample at a predefined position among the plurality of samples belonging to the first sample group may be assigned to a sample (DS1) in the downsampling region. A sample at a predefined position among the plurality of samples belonging to the second sample group may be assigned to a sample (DS2) in the downsampling region. The predefined position may represent a fixed position that is pre-determined in the encoding / decoding device. As an example, it may be any one of the leftmost, rightmost, or middle positions. In the left adjacent region, a sample at a predefined position among the plurality of samples belonging to each sample group may also be assigned to a sample in the downsampling region. In this case, the predefined position may be any one of the top layer, the bottom layer, or middle positions.

[0188] FIG. 7 is a diagram illustrating a downsampling method based on a weighted average value according to an embodiment to which the present invention can be applied.

[0189] In this embodiment, the average value may be calculated by the following formula (hereinafter referred to as the first average expression):

[0190] Sum of samples in a sample group / Number of samples Alternatively, it may be calculated by the following formula (hereinafter referred to as the second average formula): Sum(weight value x samples belonging to a sample group) / number of samples FIG. 7(a) shows a case where the sample group is composed of three samples. In this case, the weight values ​​applied to the three samples may be determined to have a ratio of 1:2:1. As shown in FIG. 7(b), when the sample group is composed of five samples, the weight values ​​may be determined to have a ratio of 1:1:4:1:1. As shown in FIG. 7(c), when the sample group is composed of six samples, the weight values ​​may be determined to have a ratio of 1:2:1:2:2:1 or 1:2:2:1:2:1 in the Z direction starting from the upper left layer. Note that, although FIG. 7(a) and FIG. 7(c) show weight values ​​applied to the upper layer adjacent region, they can be applied to the left adjacent region as well.

[0191] The average value may include a result value derived by applying a predetermined operation to a plurality of average values ​​calculated by the first average expression or the second average expression. Here, the predetermined operation may be the first average expression or the second average expression. For example, when three samples (i.e., the first to third samples) belong to a sample group, an average value (first value) of the first sample and the second sample and an average value (second value) of the second sample and the third sample can be calculated. The average value can be derived from the calculated average value of the first value and the second value.

[0192] The downsampling method may be applied only to the upper adjacent region, or conversely, only to the left adjacent region, or the downsampling method according to Fig. 6 (hereinafter referred to as the first method) may be applied to either the upper or left adjacent region, and the downsampling method according to Fig. 7 (hereinafter referred to as the second method) may be applied to the other one of the upper or left adjacent region.

[0193] In addition, at least one of the first method and the second method can be selectively used depending on the size / shape of the current block. For example, if the width of the current block is greater than a predetermined threshold, the first method can be applied to the upper adjacent region of the current block, and otherwise the second method can be applied. In a similar manner, downsampling can be performed on the height of the current block. Or, if the current block is non-square, the first method can be applied to either the upper or left adjacent region, and the second method can be applied to the other region. In this case, if the width of the current block is greater than the height, the first method can be applied to the upper adjacent region, and the second method can be applied to the left adjacent region. Conversely, if the width of the current block is less than the height, the second method can be applied to the upper adjacent region, and the first method can be applied to the left adjacent region. If the current block is square, the same downsampling method can be used in the upper and left adjacent regions. Here, the downsampling method may be limited to the first method.

[0194] FIG. 8 is a diagram illustrating a method of dividing a first predicted sample and performing interpolation for other regions according to an embodiment to which the present invention can be applied.

[0195] Referring to (a) in FIG. 8, a predicted sample of a DS block may be assigned to a predicted sample at a predetermined position in a current block. Here, the predetermined position may be determined by considering a ratio of the size of the current block to the size of the DS block. For example, the correspondence relationship between the predicted samples of the DS block and the current block may be defined by Equation 2.

[0196] [Formula 2] First predicted sample curBLK[(x+1)×r-1][(y+1)×r-1]=First predicted sample dsBLK[x][y] Here, r represents the size ratio between the current block and the DS block, and x and y are the x-axis and y-axis coordinates of the first predicted sample in each DS block, respectively. The first predicted sample curBLK may represent the position of the first predicted sample in the current block, and the first predicted sample dsBLK may represent the position of the first predicted sample in the DS block.

[0197] Regarding the interpolation, as shown in (b) of Fig. 8, at least one of the first predicted sample assigned to the current block or the reconstructed sample adjacent to the current block (hereinafter referred to as an interpolation target sample) may be used to derive a sample in the current block that is not assigned to the first predicted sample (hereinafter referred to as an interpolation target sample). Note that the interpolation reference sample may further include a predicted sample generated by interpolation before the current sample to be interpolated (i.e., the immediately preceding sample to be interpolated).

[0198] The positions and ranges of the reconstructed samples adjacent to the current block are the same as those of the reference samples, and will not be described in detail here.

[0199] According to the position of the sample to be interpolated, the interpolation reference sample may be composed of a plurality of first predicted samples, or may be composed of at least one first predicted sample and at least one reconstructed adjacent sample. The reconstructed adjacent sample may selectively use any one of the x-coordinate or y-coordinate samples that are the same as the sample to be interpolated, or may use at least one of the x-coordinate or y-coordinate and a plurality of samples of the sample to be interpolated. The selection may be made based on the position of the sample to be interpolated. For example, if the sample to be interpolated has an x-coordinate similar to the first predicted sample, the reconstructed adjacent sample may include only samples that have an x-coordinate similar to the sample to be interpolated. Conversely, if the sample to be interpolated has a y-coordinate similar to the first predicted sample, the reconstructed adjacent sample may include only samples that have a y-coordinate similar to the sample to be interpolated. Or, the reconstructed adjacent sample may include a plurality of samples located on the same horizontal and vertical lines as the sample to be interpolated.

[0200] The interpolation target sample may be derived from a representative value of a plurality of interpolation reference samples, where the representative value may include any one of an average value, a minimum value, a maximum value, a mode value, and a median value.

[0201] The average value may be calculated by the following formula (hereinafter referred to as the first average expression):

[0202] Sum of difference reference samples / Number of difference reference samples Alternatively, it may be calculated using the following formula (hereinafter referred to as the second average expression):

[0203] Sum(weight value x difference value reference sample) / number of difference value reference samples The weight value according to the second average expression can be determined based on the relative / absolute distance between the sample to be interpolated and the interpolation reference sample, which will be described in detail with reference to FIG.

[0204] FIG. 9 is a diagram illustrating the assignment of weight values ​​to distances in an interpolation step according to one embodiment to which the present invention can be applied.

[0205] The weighting value of the present invention may include a weighting value determined based on a distance from the sample to be interpolated to the interpolated reference sample. As an example, referring to FIG. 9, when the first sample to be interpolated 910 is subjected to interpolation, the ratio of the distance from the first sample to be interpolated 910 to the first interpolated reference sample 911 and the distance from the first sample to be interpolated 910 to the second interpolated reference sample 912 is 3:1, so that the ratio of the weighting values ​​applied to the first interpolated reference sample 911 and the second interpolated reference sample may be 1:3. When the second sample to be interpolated 920 is subjected to interpolation, the ratio of the distance from the second sample to be interpolated 920 to each of the first interpolated reference samples 921 and the distance from the second sample to be interpolated 920 to the second interpolated reference sample 922 is 1:1, so that the ratio of the weighting values ​​applied to the first interpolated reference sample and the second interpolated reference samples 921 and 922 may be 1:1.

[0206] The interpolation filter of the present invention may have a directional property, which may include a vertical, horizontal, z-shaped, diagonal, or other direction.

[0207] The interpolation may be performed based on a predetermined priority order. The priority order may be one of: performing interpolation in the vertical direction, followed by interpolation in the horizontal direction (first priority order), or performing interpolation in the horizontal direction, followed by interpolation in the vertical direction (second priority order), or performing interpolation in the vertical and horizontal directions simultaneously (third priority order).

[0208] The current block may be interpolated using only one of the first to third orders, or may be interpolated using a combination of at least two of the first to third orders. The interpolation orders may be understood in detail with reference to FIG. 10.

[0209] FIG. 10 is a diagram showing the order of interpolation steps according to one embodiment to which the present invention can be applied.

[0210] (a) in Figure 10 is the first order in Figure 9. Specifically, first, an interpolation is performed on a vertical line to which a first predicted sample belongs, and then an interpolation is performed on a horizontal line based on the interpolated line and the left interpolation reference sample of the current block.

[0211] Figure 10(b) is the second order in Figure 9. Specifically, first, the horizontal line to which the first predicted sample belongs is interpolated, and then, based on the interpolated line and the upper layer interpolation reference sample of the current block, the vertical line is interpolated.

[0212] (c) in FIG. 10 is the third order in FIG. 9. First, the vertical and horizontal lines to which the first predicted sample belongs are interpolated. Then, the remaining samples that are not interpolated are interpolated. In this case, only the vertical lines or the horizontal lines can be interpolated, or the vertical and horizontal lines can be interpolated simultaneously. When the vertical and horizontal lines are interpolated simultaneously, one sample to be interpolated may have a first interpolated value on the vertical line and a second interpolated value on the horizontal line at the same time. In this case, a representative value of the first and second interpolated values ​​can be assigned to the sample to be interpolated. Here, the representative value can be derived from an average value, a minimum value, a maximum value, a mode value, or a median value.

[0213] The interpolation order may be a predetermined order in the encoding / decoding device, or may be selectively determined based on the encoding information of the current block, where the encoding information is as described above, and therefore, a detailed description thereof will be omitted here.

[0214] The order may be determined based on the size of the block, which may be expressed as one of the width or height, the minimum / maximum of the width and height, the sum of the width and height, the number of samples belonging to the block, etc.

[0215] For example, if the size of the current block is greater than a predetermined threshold, the first interpolation can be performed; otherwise, the second interpolation can be performed. Conversely, if the size of the current block is less than a predetermined threshold, the second interpolation can be performed; otherwise, the first interpolation can be performed. The threshold may be 8, 16, 32, or a larger natural number.

Claims

1. 1. A method of video signal processing, performed by a decoder, comprising: determining an intra-prediction mode for a current block; determining a reference sample to be used for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode, the matrix being determined from a group of matrices; predicting the current block based on the reference samples and a matrix; The step of predicting the current block based on the reference samples and a matrix includes: applying the matrix to the reference samples to generate a prediction block; the matrix is ​​determined by further considering coding information of the current block; The coding information includes a size and a shape of the current block.

2. The step of determining a reference sample comprises: determining a neighboring region of the current block; and downsampling the determined adjacent region; The contiguous region is divided into a plurality of sample groups; The sample group consists of one or more samples, a representative value of the sample group is determined for the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value.

2. The method of claim 1.

3. The step of predicting the current block based on the reference samples and a matrix includes: The method further includes rearranging all or a part of the prediction samples of the generated prediction block.

2. The method of claim 1.

4. The step of predicting the current block based on the reference samples and a matrix includes: performing an interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block.

2. The method of claim 1.

5. 1. A method of video signal processing, performed by an encoder, comprising: determining an intra-prediction mode for a current block; determining a reference sample to be used for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode, the matrix being determined from a group of matrices; predicting the current block based on the reference samples and a matrix; The step of predicting the current block based on the reference samples and a matrix includes: applying the matrix to the reference samples to generate a prediction block; the matrix is ​​determined by further considering coding information of the current block; The coding information includes a size and a shape of the current block.

6. The step of determining a reference sample comprises: determining a neighboring region of the current block; and downsampling the determined adjacent region; The contiguous region is divided into a plurality of sample groups; The sample group consists of one or more samples, a representative value of the sample group is determined for the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value.

6. A method for processing a video signal according to claim 5.

7. The step of predicting the current block based on the reference samples and a matrix includes: The method further includes rearranging all or a part of the prediction samples of the generated prediction block.

6. A method for processing a video signal according to claim 5.

8. The step of predicting the current block based on the reference samples and a matrix includes: performing an interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block.

6. A method for processing a video signal according to claim 5.

9. A decoding device for processing a video signal, comprising: A first determination unit configured to determine an intra-prediction mode of a current block; A second determination unit configured to determine a reference sample used for intra prediction of the current block; a third determination unit configured to determine a predetermined matrix based on the intra prediction mode, the matrix being determined from a group of a plurality of matrices; a generation unit configured to generate a prediction block by applying the matrix to the reference samples; the matrix is ​​determined by further considering coding information of the current block; A decoder for processing a video signal, wherein the coding information includes a size and a shape of the current block.

10. The second determination unit further comprises: determining a neighboring region of the current block; configured to perform downsampling on the determined adjacent region; The contiguous region is divided into a plurality of sample groups; The sample group consists of one or more samples, a representative value of the sample group is determined for the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value. Decoding device for processing a video signal according to claim 9.

11. The generating unit further comprises: configured to rearrange prediction samples of all or a part of the generated prediction block. Decoding device for processing a video signal according to claim 9.

12. The generating unit further comprises: configured to perform an interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block. Decoding device for processing a video signal according to claim 9.

13. 1. An encoding device for video signal processing, comprising: A first determination unit configured to determine an intra-prediction mode of a current block; A second determination unit configured to determine a reference sample used for intra prediction of the current block; a third determination unit configured to determine a predetermined matrix based on the intra prediction mode, the matrix being determined from a group of a plurality of matrices; a generation unit configured to generate a prediction block by applying the matrix to the reference samples; the matrix is ​​determined by further considering coding information of the current block; An encoding apparatus for processing a video signal, wherein the encoding information includes a size and a shape of the current block.

14. The second determination unit further comprises: determining a neighboring region of the current block; configured to perform downsampling on the determined adjacent region; The contiguous region is divided into a plurality of sample groups; The sample group consists of one or more samples, a representative value of the sample group is determined for the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value. Encoding device for processing a video signal according to claim 13.

15. The generating unit further comprises: configured to rearrange prediction samples of all or a part of the generated prediction block. Encoding device for processing a video signal according to claim 13.

16. The generating unit further comprises: configured to perform an interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block. Encoding device for processing a video signal according to claim 13.

Images