Method and device for coding and decoding image by using intra-prediction

Abstract

To provide a method and device for processing a video signal which improves the efficiency of coding and decoding by dividing a tree block, performing an intra-prediction based on a matrix, and performing down-sampling or up-sampling used for the intra-prediction.SOLUTION: The method for processing a video signal includes: determining an intra-prediction mode of a curent block; determining a reference sample used for intra-prediction of the current block; determining a predetermined matrix on the basis of the intra-prediction mode; and predicting the current block on the basis of the reference sample and the matrix.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to an image encoding / decoding method and apparatus.

Background Art

[0002] With the spread of the Internet, portable terminals, and the growth of information and communication technologies, the use of multimedia data has been increasing rapidly. Therefore, due to image prediction in various systems, in order to execute various services or tasks, the demand for the performance and efficiency of image processing systems has been significantly increasing, but research and development results that can cope with such situations are considerably lacking.

[0003] Therefore, in the conventional image encoding / decoding method and apparatus, it is necessary to improve image processing, and in particular, it is necessary to improve image encoding or image decoding performance.

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is desirable for the present invention to improve encoding / decoding efficiency by adaptive block splitting.

[0005] It is desirable for the present invention to improve encoding / decoding efficiency by intra prediction based on a matrix.

[0006] The present invention provides a method and apparatus for determining a reference sample and a matrix used for intra prediction based on a matrix.

[0007] The present invention provides a method and apparatus for downsampling and upsampling used for intra prediction based on a matrix.

Means for Solving the Problems

[0008] The video signal processing method and apparatus according to the present invention can determine the intra prediction mode of the current block, determine the reference samples used for the intra prediction of the current block, determine a predetermined matrix based on the intra prediction mode, and predict the current block based on the reference samples and the matrix.

[0009] In the video signal processing method and apparatus according to the present invention, the step of determining the reference samples may include the step of determining the adjacent region of the current block and the step of performing downsampling on the determined adjacent region.

[0010] In the video signal processing method and apparatus according to the present invention, the adjacent region is divided into a plurality of sample groups, the sample group consists of one or more samples, the representative value of the sample group is determined by the reference sample, and the representative value may be any one of an average value, a minimum value, a maximum value, a most frequent value, or a median value.

[0011] In the video signal processing method and apparatus according to the present invention, the matrix is determined by further considering the coding information of the current block, and the coding information may include the size, shape, angle, or directionality of the intra prediction mode of the current block.

[0012] In the video signal processing method and apparatus according to the present invention, the step of predicting the current block may include the step of generating a prediction block by applying the matrix to the reference samples.

[0013] In the video signal processing method and apparatus according to the present invention, the step of predicting the current block may further include the step of rearranging all or some of the prediction samples of the generated prediction block.

[0014] In the video signal processing method and apparatus according to the present invention, the step of predicting the current block may further include performing interpolation on the current block based on at least one of the predicted block or the reconstructed samples adjacent to the current block.

Advantages of the Invention

[0015] According to the present invention, by dividing the tree block, the encoding / decoding efficiency can be improved.

[0016] According to the present invention, by intra prediction based on a matrix, the encoding / decoding efficiency can be improved.

[0017] According to the present invention, by downsampling or upsampling used in intra prediction based on a matrix, the encoding / decoding efficiency can be improved.

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

DETAILED DESCRIPTION OF THE INVENTION

[0019] The video signal processing method and apparatus according to the present invention can determine the intra prediction mode of the current block, determine the reference samples used for the intra prediction of the current block, determine a predetermined matrix based on the intra prediction mode, and predict the current block based on the reference samples and the matrix.

[0020] In the video signal processing method and apparatus according to the present invention, the step of determining the reference samples may include a step of determining an adjacent region of the current block and a step of performing downsampling on the determined adjacent region.

[0021] In the video signal processing method and apparatus according to the present invention, the adjacent region is divided into a plurality of sample groups, the sample group consists of one or more samples, the representative value of the sample group is determined by the reference sample, and the representative value may be any one of an average value, a minimum value, a maximum value, a most frequent value, or a median value.

[0022] In the video signal processing method and apparatus according to the present invention, the matrix is determined by further considering the encoding information of the current block, and the encoding information may include the size, shape, angle, or directionality of the intra prediction mode of the current block.

[0023] In the video signal processing method and apparatus according to the present invention, the step of predicting a current block may include a step of generating a prediction block by applying the matrix to the reference sample.

[0024] In the video signal processing method and apparatus according to the present invention, the step of predicting a current block may further include a step of rearranging all or some of the prediction samples of the generated prediction block.

[0025] In the video signal processing method and apparatus according to the present invention, the step of predicting a current block may further include performing interpolation on the current block based on at least one of the prediction block or the reconstructed samples adjacent to the current block.

[0026] In the present invention, various modifications are possible and various embodiments can be provided. Specific embodiments are shown in the drawings and described in detail. However, these specific embodiments do not limit the present invention to specific embodiments, but include all modifications, equivalents or alternatives included in the spirit and technical scope of the present invention. When explaining each drawing, similar reference numerals are used for similar components.

[0027] Terms such as first, second, etc. may be used to describe components, but the components are not limited to these terms. These terms are only used to distinguish one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may also be named the first component. The term "and / or" refers to a combination of a plurality of described related items or any one of the plurality of described related items.

[0028] When describing that one component is "connected" or "coupled" to another component, it should be understood that it may be directly connected or coupled to the other component, or there may be other components between the component and the other component. Conversely, when describing that one component is "directly connected" or "directly coupled" to another component, it should be understood that there are no other components between the component and the other component.

[0029] The terms used in this application are only for explaining specific embodiments and do not limit the present invention. Unless otherwise clearly stated in the specification, singular expressions include plural expressions. In this application, terms such as "comprising" or "having" are used to specify the existence of the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should not be construed as precluding the existence or addition possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] Unless otherwise defined, all terms including technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. Terms defined in commonly used dictionaries should be understood to have a meaning consistent with the meaning in the relevant technical literature, and should not be interpreted as having an ideal or formal meaning unless clearly defined herein.

[0031] Hereinafter, with reference to the drawings, preferred embodiments of the present invention will be described in more detail. Hereinafter, for the same components in the drawings, the same reference numerals will be used, and duplicate descriptions of the same components will be omitted.

[0032] FIG. 1 is a schematic block diagram showing an encoding device according to an embodiment of the present invention.

[0033] Referring to FIG. 1, the encoding device 100 may include an image segmentation unit 110, prediction units 120, 125, a conversion unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse conversion unit 145, a filtering unit 150, and a memory 155.

[0034] Each component shown in FIG. 1 is shown alone and represents different characteristic functions in the image encoding device. Also, each component may represent being composed of separate hardware. However, for the sake of convenience in explanation, each component is described as such and includes it. At least two of the components may constitute one component, or one component may be divided into a plurality of components to execute functions. Such comprehensive examples and individual examples of each component are also included within the scope of the rights of the present invention as long as they do not depart from the essence of the present invention.

[0035] Note that some components may not be essential components for executing the basic functions in the present invention, but may only be selectable components for improving performance. The present invention may also be realized by including only the components. The components are essential for realizing the essence of the present invention other than the components for improving performance. A structure including only the essential components other than the selectable components for improving normality is also included within the scope of the rights of the present invention.

[0036] The image segmentation unit 110 can divide the input image into at least one block. In this case, the block may represent a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The division can be performed by at least one of a quad tree, a binary tree, and a ternary tree. A quad tree is a method of dividing an upper-layer block into four lower-layer blocks whose width and height are half of the width and height of the upper-layer block. A binary tree is a method of dividing the upper-layer block into two lower-layer blocks having at least one width or height that is half of the upper-layer block. In the division of the binary tree, based on the division based on the binary branch having half the height of the upper-layer block, the block may have not only a square shape but also a non-square shape.

[0037] Hereinafter, in the embodiments of the present invention, the coding unit may be used for a unit that performs coding or a unit that performs decoding.

[0038] The prediction units 120 and 125 may include an inter-prediction unit 120 for performing inter-prediction and an intra-prediction unit 125 for performing intra-prediction. For the prediction unit, it can be determined whether to use frame prediction or intra-prediction, and specific information (such as intra-prediction mode, motion vector, reference image, etc.) according to each prediction method can also be determined. In this case, the processing unit for executing the prediction may be different from the processing unit for determining the prediction method and specific content. For example, the prediction method and prediction mode, etc., may be determined by the prediction unit, and the prediction may be executed by the conversion unit. The residual value (residual block) between the generated prediction block and the original block may be input to the conversion unit 130. Note that in the entropy encoding unit 165, prediction mode information, motion vector information, etc., used for prediction can be encoded together with the residual value and transmitted to the decoding device. When using one encoding mode, it is not necessary to generate a prediction block by the prediction units 120 and 125, and similarly, the original block can be encoded and transmitted to the decoding unit.

[0039] The inter-prediction unit 120 can predict the prediction unit based on information of at least one of the image immediately before or immediately after the current image. In some cases, the prediction unit can also be predicted based on information of some regions within the current block where encoding has been completed. The prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0040] In the reference image interpolation section, the reference image information is provided by the memory 155. Also, in the reference image, pixel information below integer pixels can be generated. For luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information below integer pixels in 1 / 4-pixel units. For chrominance signals, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information below integer pixels in 1 / 8-pixel units.

[0041] The motion prediction section can perform motion prediction based on the reference image interpolated by the reference image interpolation section. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can all be used as methods for calculating motion vectors. According to the interpolated pixels, the motion vector may have a motion vector value in 1 / 2 or 1 / 4-pixel units. The motion prediction section can predict the current prediction unit by different motion prediction methods. Various methods such as the Skip method, the Merge method, and the Advanced Motion Vector Prediction (AMVP) method can be used as motion prediction methods.

[0042] The intra prediction unit 125 can generate a prediction unit based on reference pixel information adjacent to the current block. Here, the reference pixel information adjacent to the current block is pixel information in the current image. Since the adjacent (peripheral) blocks of the current prediction unit are inter-predicted blocks, when the reference pixel is an inter-predicted pixel, the reference pixel information included in the inter-predicted block can be replaced with the reference pixel information of the adjacent intra-predicted block. That is, when the reference pixel is unavailable, the unavailable reference pixel information can be replaced with at least one of the available reference pixels.

[0043] The prediction mode in intra prediction may include a directional prediction mode that uses reference pixel information based on the prediction direction and a non-directional mode that does not use directional information during prediction. The mode for predicting the luminance component may be different from the mode for predicting the chrominance component. Also, the chrominance component can be predicted using the intra prediction mode used to predict the luminance component or the predicted / reconstructed (restored) luminance component.

[0044] The intra prediction method can generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to reference pixels based on the intra prediction mode. The type of the AIS filter applied to the reference pixels may be different. To execute the intra prediction method, the intra prediction mode of the current prediction unit can be predicted from the intra prediction modes located in prediction units adjacent to the current prediction block. When predicting the prediction mode of the current prediction unit using the mode information predicted from adjacent prediction units, if the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the adjacent prediction unit, predetermined flag information can be used to transmit information indicating that the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the adjacent prediction unit. If the intra prediction mode of the current prediction unit is different from the intra prediction mode of the adjacent prediction unit, entropy coding can be performed to encode the intra prediction mode information of the current block.

[0045] In addition, in the prediction units 120 and 125, residual value (Residual) information including the difference value between the generated prediction unit and the original block can be generated. The generated residual block can be input to the conversion unit 130.

[0046] In the conversion unit 130, the residual block including residual data can be converted using a conversion type such as DCT or DST. In this case, the conversion type can be determined based on the intra prediction mode of the prediction unit used to generate the residual block.

[0047] The quantization unit 135 can perform quantization on the values converted to the frequency domain by the conversion unit 130. The quantization coefficient may vary according to the importance of the block or the image. The values calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0048] The rearrangement unit 160 can rearrange the coefficient values for the quantized residual block. The rearrangement unit 160 can change the coefficients in a two-dimensional block shape into a one-dimensional vector shape by a coefficient scanning method. For example, the rearrangement unit 160 can scan from the DC coefficient to the coefficients in the high-frequency region using a predetermined scanning type and change them into a one-dimensional vector shape.

[0049] The entropy encoding unit 165 can perform entropy encoding based on the value calculated by the rearrangement unit 160. Entropy encoding can use various encoding methods such as the exponential Golomb coding method, CAVLC (Context-Adaptive Variable Length Coding), and context-adaptive binary arithmetic coding (CABAC).

[0050] The entropy encoding unit 165 can perform encoding on various information from the rearrangement unit 160 and the prediction units 120 and 125, such as the residual coefficient information and block type information of the coding unit, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference image information, block interpolation information, filtering information, etc.

[0051] The entropy encoding unit 165 can perform entropy encoding on the coefficient values of the coding unit input to the rearrangement unit 160.

[0052] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the values quantized by the quantization unit 135 and perform inverse transformation on the values transformed by the transform unit 130. The residual values (Residual) generated by the inverse quantization unit 140 and the inverse transform unit 145 can be merged with the predicted prediction units by the motion estimation unit, the motion compensation unit, and the intra prediction unit included in the prediction units 120 and 125 to generate a reconstructed block (Reconstructed Block).

[0053] The filtering unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0054] The deblocking filter can remove the block distortion caused by the boundary between blocks in the reconstructed image. To determine whether to perform deblocking, it is determined whether to apply the deblocking filter to the current block based on the pixels included in some columns or rows included in the block. When applying the deblocking filter to a block, a strong filter or a weak filter can be applied based on the required deblocking filter strength. When applying the deblocking filter, when performing vertical filtering and horizontal filtering, the horizontal filtering and the vertical filtering can be processed in parallel.

[0055] The offset correction unit can correct the pixel-by-pixel offset between the deblocked image and the original image. To perform offset correction on a specific image, after dividing the pixels included in the image into a certain number of regions, the region to be offset is determined, and a method of applying the offset to the region can be used, or a method of applying the offset by considering the edge information of each pixel can be used.

[0056] ALF (Adaptive Loop Filtering) can be performed based on the comparison value between the filtered reconstructed image and the original image. After dividing the pixels included in the image into predetermined groups, one filter applicable to the combination is determined. Also, for each group, filtering can be performed differently. Regarding information on whether ALF can be applied, each coding unit (CU) can transmit the luminance signal. Also, the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Note that an ALF filter having the same shape (fixed shape) can be applied regardless of the characteristics of the block to be applied.

[0057] The memory 155 can store the reconstructed block or image calculated by the filtering unit 150. The reconstructed block or image stored when performing inter prediction can be provided to the prediction units 120 and 125.

[0058] FIG. 2 is a schematic block diagram showing a decoding apparatus according to an embodiment of the present invention.

[0059] Referring to FIG. 2, the decoding apparatus 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filtering unit 240, and a memory 245.

[0060] Each component shown in FIG. 2 is shown alone and represents different characteristic functions in the image encoding apparatus, and each component may represent being configured by separate hardware. However, for convenience of explanation, each component is described as each component and includes it. At least two of the components may form one component or one component may be divided into a plurality of components to execute a function. Such comprehensive examples and individual examples of each component are also included within the scope of the present invention as long as they do not depart from the essence of the present invention.

[0061] The entropy decoding unit 210 can perform entropy decoding on the input code stream. For example, various methods such as the Exponential Golomb coding method, CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), etc. can be applied to perform entropy decoding.

[0062] The entropy decoding unit 210 can decode the information related to intra prediction and inter prediction performed by the encoding device.

[0063] The rearrangement unit 215 can rearrange the code stream entropy-decoded by the entropy decoding unit 210. It can reconstruct and rearrange the coefficients represented in the one-dimensional vector format into the coefficients in the two-dimensional block format. The rearrangement unit 215 can receive the information related to the coefficient scanning performed by the encoding device and perform rearrangement by the method of performing reverse scanning according to the order of the scanning performed by the encoding device.

[0064] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameter and the coefficient values of the rearranged blocks.

[0065] The inverse transform unit 225 can perform an inverse transform on the inverse quantized transform coefficients with a predetermined transform type. In this case, the transform type can be determined based on at least one of the information related to the prediction mode (inter / intra prediction), block size / shape, intra prediction mode, component type (luminance / chrominance component), division type (QT, BT, TT, etc.).

[0066] The prediction units 230, 235 can generate related information based on the prediction blocks provided by the entropy decoding unit 210, and can generate prediction blocks based on the previously decoded blocks or image information provided by the memory 245.

[0067] The prediction units 230 and 235 may include a prediction unit discrimination unit, an inter-prediction unit, and an intra-prediction unit. The prediction unit discrimination unit receives various types of information such as prediction unit information input by the entropy decoding unit 210, intra-prediction mode related information of the intra-prediction method, and motion prediction related information of the inter-prediction method, and in the current coding unit (CU), divides the prediction unit and discriminates whether the prediction unit performs inter-prediction or intra-prediction. The inter-prediction unit 230 can perform inter-prediction on the current prediction unit based on information included in at least one of the immediately preceding image or the immediately following image of the current image including the current prediction unit, using the information necessary for the inter-prediction of the current prediction unit provided by the encoding device. Alternatively, in the current image including the current prediction unit, inter-prediction can be performed based on information of some reconstructed regions. Therefore, the several reconstructed regions may be added to the reference image list.

[0068] In order to perform inter-prediction, based on the coding unit, it can be determined which of the skip mode, merge mode, AMVP mode, and current image reference mode is the motion prediction method of the prediction unit included in the coding unit.

[0069] The intra prediction unit 235 can generate a prediction block based on the pixel information in the current image. When the prediction unit is a prediction unit that performs intra prediction, it can perform intra prediction based on the intra prediction mode information of the prediction unit provided by the encoding device. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter, as a member that filters the reference pixels of the current block, can determine whether to apply the filter based on the prediction mode of the current prediction unit. Using the prediction mode of the prediction unit and the AIS filter information provided by the encoding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode that does not perform AIS filtering, it may not be necessary to apply the AIS filter.

[0070] When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the values of the pixels that interpolate the reference pixels, the reference pixel interpolation unit can interpolate the reference pixels and generate reference pixels in pixel units less than or equal to an integer value. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixels, it may not be necessary to interpolate the reference pixels. When the prediction mode of the current block is the DC mode, the DC filter can generate a prediction block by filtering.

[0071] The reconstructed block or image may be provided to the filtering unit 240. The filtering unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0072] The deblocking filter information indicating whether to apply the deblocking filter to the block or the image can be received from the symbolization device. Further, when the deblocking filter is applied, information indicating whether a strong filter or a weak filter is applied can be received. The deblocking filter of the decoding device can receive the deblocking filter related information provided by the symbolization device. The decoding device can perform deblocking filtering on the block.

[0073] When performing encoding, the offset correction unit can perform offset correction on the reconstructed image based on the offset correction type and offset value information applied to the image.

[0074] ALF may be applied to the encoding unit based on the information indicating whether to apply ALF and the ALF coefficient information provided by the encoder. Such ALF information may be provided by being included in a specific parameter set.

[0075] The memory 245 can store the reconstructed image or block as a reference image or reference block, and can also provide the reconstructed image to the output unit.

[0076] FIG. 3 is a diagram showing a block division type according to an embodiment to which the present invention can be applied.

[0077] Referring to FIG. 3, blocks a to s can be obtained based on the division setting and division method, and an additional block shape not shown can also be obtained.

[0078] As an example (1), it is acceptable to use asymmetric splitting for tree-based splitting. For example, for a binary tree, blocks such as b and c may be used, or blocks such as b to g may be used. When the flag indicating the allowance of asymmetric splitting is not explicitly or implicitly activated based on the encoding / decoding setting, the acquirable candidate blocks may be b or c. When the flag indicating the allowance of asymmetric splitting is activated, the acquirable candidate blocks may be b, d, e (horizontal splitting in this example), or c, f, g (vertical splitting in this example).

[0079] In the above example, the case where the left:right or up:down length ratio of the asymmetric splitting is 1:3 or 3:1 will be described, but it is not limited to this. Depending on the encoding setting, other ratio candidate groups (for example, 1:2, 1:4, 2:3, 2:5, 3:5, etc.) may exist.

[0080] Hereinafter, various examples related to the splitting information generated in the binary tree splitting (candidate groups of 1:1, 1:3, and 3:1 in this example) will be shown.

[0081] For example, in addition to the flag indicating whether to perform splitting and the flag indicating the splitting direction, a flag indicating the splitting type can also be generated. In this case, the splitting type may represent symmetric or asymmetric splitting. Here, when the asymmetric splitting is determined as the splitting type, a flag indicating the splitting ratio can be generated. Also, based on a predetermined candidate group, an index can be assigned. When supporting a splitting ratio of 1:3 or 3:1 as a candidate group, the splitting ratio can be selected by a 1-bit flag.

[0082] Alternatively, in addition to the flag indicating whether to perform splitting and the flag indicating the splitting direction, a flag indicating the splitting ratio can also be generated. In this example, as a candidate group for the splitting ratio, a candidate having a symmetric ratio of 1:1 may be included.

[0083] In the present invention, assuming that the binary tree division has a structure as in the above example when a flag indicating permission for asymmetric division is activated. Unless otherwise explained, a binary tree represents a symmetric binary tree.

[0084] As an example (2), tree-based division can allow additional tree divisions. For example, division can be performed on a ternary tree, a quad type tree, an octa tree, etc., and thereby, n divided blocks (3, 4, 8, n in this example are integers) can be obtained. For a ternary tree, the supported blocks (when divided into a plurality of blocks in this example) may be h to m. For a quad tree, the supported blocks may be n to p. For an octa tree, the supported block may be q. Based on the encoding / decoding settings, it is possible to implicitly determine whether to support the tree-based division or explicitly generate related information. Note that based on the encoding / decoding settings, it may be used alone or mixed with a binary tree, a quad tree division, etc.

[0085] For example, for a binary tree, blocks such as b and c can be used. When mixing a binary tree and a ternary tree (assuming that in this example, the usage range of the binary tree and the usage range of the ternary tree partially overlap), blocks such as b, c, i, and l can also be used. Based on the encoding / decoding settings, when a flag indicating permission for additional division other than the current tree is explicitly or implicitly deactivated, the obtainable candidate blocks may be b or c. When activated, the obtainable candidate blocks may be b, i or b, h, i, j (horizontal division in this example), or c, l or c, k, l, m (vertical division in this example).

[0086] In the above example, it is assumed and described that the length ratio of left: middle: right or up: middle: down of the ternary tree division is 2:1:1, 1:2:1, or 1:1:2, but it is not limited thereto, and other ratios can also be set based on the encoding settings.

[0087] An example of the segmentation information generated by ternary tree segmentation (a candidate of 1:2:1 in this example) is shown below.

[0088] For example, in addition to the flag indicating whether to perform segmentation and the flag indicating the segmentation direction, a flag indicating the segmentation type can also be generated. In this case, the segmentation type may represent binary tree or ternary tree segmentation.

[0089] In the present invention, adaptive encoding / decoding settings can be applied based on the segmentation method.

[0090] As an example, the segmentation method can be determined based on the type of block. For example, the encoding block and the conversion block can use the quadtree segmentation method, and the prediction block can use the quadtree and binary tree (or ternary tree, etc.) segmentation methods.

[0091] As an example, the segmentation method can be determined based on the size of the block. For example, in some ranges between the maximum value and the minimum value of the block (for example, a×b~c×d. If the latter is larger), the quadtree segmentation method can be used. In some ranges (for example, e×f~g×h), binary tree (or ternary tree, etc.) segmentation can be performed. In this case, the range information according to the segmentation method can be explicitly generated or implicitly determined. It can also be used when the above ranges overlap.

[0092] As an example, the segmentation method can be determined based on the shape of the block (or the block before segmentation). For example, when the block shape is square, quadtree or binary tree (or ternary tree, etc.) segmentation can be performed. Or, when the shape of the block is rectangular, segmentation based on a binary tree (or ternary tree, etc.) can be performed.

[0093] As an example, the splitting setting can be determined based on the type of block. For example, in a tree-based split, the encoding block and the prediction block can use a quadtree split, and the transform block can use a binary tree split. Alternatively, the allowable split depth for the encoding block may be set to m. The allowable split depth for the prediction block may be set to n, and the allowable split depth for the transform block may be set to o. m, n, and o may be the same or different.

[0094] As an example, the splitting setting can be determined based on the size of the block. For example, in some ranges of the block (e.g., a×b~c×d), a quadtree split can be used, and in some ranges (e.g., e×f~g×h. In this example, assume that c×d is larger than g×h), a binary tree split can be used, and in some ranges (e.g., i×j~k×l. In this example, assume that g×h is greater than or equal to k×l), a ternary tree split can be used. In this case, the range may include all ranges between the maximum and minimum values of the block. The ranges may have a non-overlapping setting or an overlapping setting with each other. For example, the minimum value of some ranges may be the same as the maximum value of some ranges, or the minimum value of some ranges may be smaller than the maximum value of some ranges. When having ranges that overlap with each other, the splitting method with the larger maximum value may have a priority or information regarding which splitting method to use can be explicitly generated. That is, in the splitting method with a priority, based on the splitting result, it can be determined whether to perform the splitting method with a lower priority, or based on the splitting method selection information, it can be determined which splitting method to use.

[0095] As an example, based on the shape of the block, the splitting setting can be determined. For example, when the shape of the block is square, a quadtree split can be used. Alternatively, when the shape of the block is rectangular, a binary tree or ternary tree split can be used.

[0096] As an example, the division setting can be determined based on encoding / decoding information (e.g., slice type, color component, encoding mode, etc.). For example, when the slice type is I, the quadtree (or binary tree, ternary tree) division can be used within several ranges (e.g., a×b~c×d), when the slice type is P, it can be used within several ranges (e.g., e×f~g×h), and when the slice type is B, it can be used within several ranges (e.g., i×j~k×l). Note that when the slice type is I, the allowable depth of the quadtree (or binary tree, ternary tree division) can be set to m, when the slice type is P, the allowable depth can be set to n, and when the slice type is B, the allowable depth can be set to o. m, n, and o may be the same or different. Some slice types may have the same settings as other slices (e.g., slices P and B).

[0097] As another example, when the color component is the luminance component, the allowable depth of the quadtree (or binary tree, ternary tree) division can be set to m, when the color component is the chrominance component, it can be set to n, and m and n may be the same or different. Note that when the color component is the luminance component, the range of the quadtree (or binary tree, ternary tree) division (e.g., a×b~c×d) may be the same as or different from the range of the quadtree (or binary tree, ternary tree) division when the color component is the chrominance component (e.g., e×f~g×h).

[0098] As another example, when the encoding mode is Intra, the allowable depth of the quadtree (or binary tree, ternary tree) division can be m, when the encoding mode is Inter, it can be n (in this example, assuming n is greater than m), and m and n may be the same or different. Note that the range of the quadtree (or binary tree, ternary tree) division when the encoding mode is Intra may be the same as or different from the range of the quadtree (or binary tree, ternary tree) division when the encoding mode is Inter.

[0099] Regarding the above example, information regarding whether to support an adaptive segmentation candidate group structure based on encoding / decoding information can be explicitly generated or implicitly determined.

[0100] The above example described determining a segmentation method and segmentation settings based on encoding / decoding settings. The above example showed several situations for each element, but there may be variations in other cases. Note that the segmentation method and segmentation settings can also be determined based on a combination of multiple elements. For example, the segmentation method and segmentation settings can be determined based on the block type, size, shape, encoding / decoding information, etc.

[0101] Note that it is possible to implicitly determine elements related to the segmentation method, settings, etc. in the above example, or explicitly generate information, and determine whether to allow an applicable situation as in the above example.

[0102] The segmentation depth in the above segmentation settings represents the number of times of performing spatial segmentation based on the initial block (the segmentation depth of the initial block in this example is 0). The larger the segmentation depth, the smaller the blocks that can be divided. Based on this, depth-related settings can be set differently according to the segmentation method. For example, in the case of a tree-based segmentation method, the same depth as that of a ternary tree can be shared as the segmentation depth of a binary tree. A different depth from that of the binary tree can be used as the segmentation depth of a quaternary tree. Based on the tree type, each depth can be used.

[0103] In the above example, when using respective segmentation depths based on the tree type, the segmentation depth at the start position of the tree segmentation (the block before segmentation in this example) can be set to 0. The segmentation depth can be calculated centered on the segmentation start position without depending on the segmentation range of each tree (the maximum value in this example).

[0104] FIG. 4 is a diagram showing a block segmentation method based on a tree structure according to an embodiment to which the present invention can be applied.

[0105] In the drawings, the thick solid line represents the basic encoding block, the thick dotted line represents the quadtree division limit, the double solid line represents the symmetric binary tree division limit, the solid line represents the ternary tree division limit, and the thin solid line represents the asymmetric binary tree division limit. Except for the thick solid line, all the others represent the limits for performing division by each division method. The division settings described below (for example, division type, division information, division information arrangement order, etc.) are not limited to those in this example and may have various variations.

[0106] For the sake of convenience of explanation, it is assumed that the upper left, upper right, lower left, and lower right blocks (N×N, 64×64) based on the basic encoding block (2N×2N, 128×128) have their respective block division settings and will be described. First, through a single division operation in the initial block (the division depth is 0->1. That is, the division depth increases by 1), 4 sub-blocks are obtained. Also, in the quadtree division setting, it is assumed that the maximum encoding block is 128×128, the minimum encoding block is 8×8, and the maximum division depth is 4. This is a setting commonly applied to each block.

[0107] (Section 1. Upper left block, A1~A6) In this example, when supporting the division of the single tree method (the quadtree in this example), the size and shape of the blocks that can be obtained can be determined by one block division setting such as the maximum encoding block, minimum encoding block, division depth, etc. In this example, when there is one block that can be obtained by division (performing 2 divisions in the horizontal and vertical directions respectively), the division information required for a single division operation (based on the block 4M×4N before division, the division depth increases by 1) is a flag indicating whether to perform division (in this example, if it is 0, no division is performed, and if it is 1, division is performed), and the possible candidates that can be obtained may be 4M×4N and 2M×2N.

[0108] (Section 2. Upper right block, A7~A11) In this example, when supporting multiple tree - based partitions (quad - trees and binary - trees in this example), the size and shape of the blocks that can be obtained can be determined by multiple block - splitting settings. In this example, assume that the maximum - coded block for a binary - tree is 64×64, and the minimum - coded block has a length of 4 and a maximum splitting depth of 4.

[0109] In this example, when there are two blocks that can be obtained by splitting (two or four in this example), the splitting information required for one splitting operation (the quadtree splitting depth increases by 1) is a flag indicating whether to perform the splitting, a flag indicating the splitting type, a flag indicating the splitting type, and a flag indicating the splitting direction. The possible candidates that can be obtained are 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N.

[0110] When the quadtree splitting range and the binary - tree splitting range overlap (that is, the range where the quadtree splitting and the binary - tree splitting can be executed in the current step), and the current block (the state before splitting) is a block obtained by quadtree splitting (a block obtained by quadtree splitting in the parent block <when the splitting depth is 1 less than the current one>), classification can be performed as follows, and the splitting information can be arranged. That is, when the blocks supported by each splitting setting can be obtained by multiple splitting methods, classification can be performed in the following way, and the splitting information can be generated.

[0111] (1) When the quadtree splitting and the binary - tree splitting overlap

Table 1

[0112] In the above table, a is a flag indicating whether to perform quadtree splitting, and 1 indicates that quadtree splitting (QT) is to be performed. If the flag is 0, then flag b, which indicates whether to perform binary tree splitting, is checked. If b is 0, then no splitting is performed on the block (No Split). If b is 1, then binary tree splitting is performed.

[0113] c is a flag indicating the splitting direction. If c is 0, then horizontal splitting (hor) is performed. If c is 1, then vertical splitting (ver) is performed. d is a flag indicating the splitting type. If d is 0, then symmetric splitting (Symmetric Binary Tree: SBT) is performed. If d is 1, then asymmetric splitting (Asymmetric Binary Tree: ABT) is performed. Only when d is 1, information regarding the detailed splitting ratio in asymmetric splitting (1 / 4 or 3 / 4) is checked. If d is 0, then in the left / right blocks or up / down blocks, the ratio of the left block to the upper block is 1 / 4, and the ratio of the right block to the lower block is 3 / 4. If d is 1, it is the reverse.

[0114] (2) When only binary tree splitting is executable, In the above table, the splitting information can be represented by flags b to e other than a.

[0115] Regarding block A7 in Figure 4, since quadtree splitting can be performed on the block before splitting (A7 to A11) (that is, quadtree splitting can be performed, but binary tree splitting is performed without performing quadtree splitting), it belongs to the case of generating the splitting information in (1).

[0116] Conversely, regarding A8 to A11, since binary tree splitting was performed on the block before splitting (A8 to A11) without performing quadtree splitting (that is, when quadtree splitting cannot be further performed on the block <A8 to A11>), it belongs to the case of generating the splitting information in (2).

[0117] (3rd, lower left block, A12 to A15) In this example, when supporting the splitting of the multiple-tree method (quad-tree, binary tree, and ternary tree in this example), the size and shape of the blocks that can be obtained can be determined by multiple block splitting settings. In this example, assume that the maximum encoded block for the binary tree / ternary tree is 64×64, and the minimum encoded block has a length of 4 and a maximum splitting depth of 4.

[0118] In this example, when there are two or more blocks that can be obtained by splitting (2, 3, or 4 in this example), the splitting information required for a single splitting operation is a flag indicating whether to perform splitting, a flag indicating the splitting type, and a flag indicating the splitting direction. The available candidates may be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×2N / 4M×N, M×4N / 2M×4N / M×4N.

[0119] When the quadtree splitting range and the binary tree / ternary tree splitting range overlap, and the current block is a block obtained by quadtree splitting, the classification can be performed as follows, and the splitting information can be arranged.

[0120] (1) When the quadtree splitting and the binary tree / ternary tree splitting overlap

Table 2

[0121] In the above table, a is a flag indicating whether to perform quadtree splitting. When it is 1, it represents performing quadtree splitting. When the flag is 0, check the flag b indicating whether to perform binary tree or ternary tree splitting. When b is 0, no splitting is performed on the block. When b is 1, binary tree or ternary tree splitting is performed.

[0122] c is a flag indicating the splitting direction. When c is 0, horizontal splitting is performed. When c is 1, vertical splitting is performed. d is a flag indicating the splitting category. When d is 0, binary tree splitting (BT) is performed. When d is 1, ternary tree splitting (TT) is performed.

[0123] (2) When only binary tree / ternary tree splitting is possible In the said table, from b to d which are flags other than a, the splitting information can be represented.

[0124] Regarding blocks A12 and A15 in Figure 4, since a quadtree split can be performed on the blocks (A12 - A15) before splitting, it belongs to the case of generating the splitting information in (1).

[0125] Conversely, regarding A13 and A14, since a ternary tree split was performed on the blocks (A13, A14) before splitting without performing a quadtree split, it belongs to the case of generating the splitting information in (2).

[0126] (Fourth, bottom - left block, A16 - A20) In this example, as a case of supporting splitting of multiple - tree methods (quadtree, binary tree, ternary tree in this example), the size and shape of the obtainable blocks can be determined by multiple block - splitting settings. In this example, assume that the maximum encoded block for the binary tree / ternary tree is 64×64, the minimum encoded block has a length of 4, and the maximum splitting depth is 4.

[0127] In this example, when there are two or more blocks obtainable by splitting (2, 3, 4 in this example), the splitting information required for one splitting operation is a flag indicating whether to perform splitting, a flag indicating the splitting category, a flag indicating the splitting type, and a flag indicating the splitting direction, and the obtainable candidates may be 4M×4N, 4M×2N, 2M×4N, 4M×N / 4M×3N, 4M×3N / 4M×N, M×4N / 3M×4N, 3M×4N / M×4N, 4M×N / 4M×2N / 4M×N, M×4N / 2M×4N / M×4N.

[0128] When the quadtree splitting range and the binary - tree / ternary - tree splitting range overlap and the current block is a block obtained by quadtree splitting, the classification can be performed as follows, and the splitting information can be arranged.

[0129] (1) When the quadtree division and the binary tree / ternary tree division overlap

Table 3

[0130] In the said table, a is a flag indicating whether to perform quadtree division. If it is 1, quadtree division is performed. If the said flag is 0, a flag b indicating whether to perform binary tree division is confirmed. If b is 0, no further division is performed in the block. If b is 1, binary tree or ternary tree division is performed.

[0131] c is a flag indicating the division direction. If c is 0, horizontal division is performed. If c is 1, vertical division is performed. d is a flag indicating the division category. If d is 0, ternary tree division is performed. If d is 1, binary tree division is performed. If d is 1, a flag e indicating the division type is confirmed. If e is 0, symmetric division is performed. If e is 1, asymmetric division is performed. If e is 1, information on the detailed division ratio in asymmetric division is confirmed. This is the same as the previous example.

[0132] (2) When only binary tree / ternary tree division is executable In the said table, the division information can be represented by flags b to f other than a.

[0133] Regarding block A20 in FIG. 4, since quadtree division can be performed on the block before division (A16 to A19), it belongs to the case of generating the division information in (1).

[0134] Conversely, regarding A16 to A19, since binary tree division was performed without performing quadtree division on the block before division (A16 to A19), it belongs to the case of generating the division information in (2).

[0135] FIG. 5 is a diagram showing the process of performing intra prediction on the current block based on a matrix according to an embodiment of the present invention.

[0136] Referring to FIG. 5, the intra prediction mode currently used for intra prediction of a block can be determined (S500).

[0137] When performing intra prediction of the current block, the encoding / decoding apparatus can determine the intra prediction mode. The current block may be an encoding block (CU), a prediction block (PU), a transform block (TU), or any one of these blocks.

[0138] (Embodiment 1) Based on the information for transmitting a signal, the intra prediction mode can be determined. The information can specify any one of N predefined intra prediction modes in the encoding / decoding apparatus. The predefined intra prediction modes represent all the intra prediction modes available for the current block. N may be a natural number of 67 or less and 11 or more (for example, 67, 35, 11). Note that the value of N can be determined based on the size of the current block. For example, when the current block is less than 8×8, N is determined to be 35. Otherwise, N may be determined to be either 19 or 11.

[0139] (Embodiment 2) The intra prediction mode can also be determined by a default mode or an index predefined in the encoding / decoding apparatus. The default mode may be at least one of Planar mode (index 0), DC mode (index 1), horizontal mode (index 18), vertical mode (index 50), diagonal mode (index 2, 34, 66). Here, the index corresponds to the case where there are 67 predefined intra prediction modes, and different indexes can be assigned for each mode based on the value of N.

[0140] (Embodiment 3) Based on the encoded information, the intra prediction mode can be variably determined. Here, the encoded information may include not only information for transmitting a signal encoded by an encoding device, but also information derived based on information for transmitting a signal by a decoding device. The encoded information may be information related to at least one of the current block or an adjacent block. The adjacent block may include a spatially and / or temporally adjacent block of the current block, and the spatially adjacent block may represent a block adjacent to at least one of the left side, upper layer, upper left layer, lower left layer, or upper right layer of the current block.

[0141] The encoded information may include block size / shape, block availability, split type, number of splits, component type, prediction mode, information related to intra prediction mode, inter mode, motion information, transform type, transform skip mode, information related to non-zero residual coefficients, scan order, color format, loop filter information, etc. The block size may be represented by either the width or the height, the minimum / maximum value in width and height, the sum of width and height, the number of samples belonging to the block, etc. The availability of the block can be determined by considering the block position, the range of the parallel processing area, the decoding order, etc. The prediction mode can represent information for indicating an intra mode or an inter mode. The information related to the intra prediction mode may include information related to whether the intra prediction mode is a non-directional mode, whether the intra prediction mode is a vertical / horizontal mode, the directionality of the intra prediction mode, the number of intra prediction modes predefined in the encoding / decoding device, etc. The inter mode can represent information for indicating a merge / skip mode, an AMVP mode, or a current picture reference mode. The current picture reference mode represents a method of predicting the current block using the reconstructed area of the current picture. The current picture may be the picture to which the current block belongs. Inter prediction can be performed by adding the current picture to the reference picture list. The current picture may be arranged after the short-term or long-term reference pictures in the reference picture list. The motion information may include a prediction direction flag, a motion vector, a reference picture index, etc.

[0142] (Example 4) The intra prediction mode can also be derived based on the MPM list and the MPM index. The MPM list includes a plurality of MPMs. The MPM can be determined based on the intra prediction modes of the spatial / temporal adjacent blocks of the current block. The number of MPMs is x, and x may be 3, 4, 5, 6, or an integer greater than that.

[0143] For example, the MPM list may include at least one of the intra prediction modes mode A, (mode A-n), (mode A+n) of adjacent blocks, or the default mode. The value of n may be 1, 2, 3, 4, or an integer greater than that. The adjacent block can represent an adjacent block on the left side and / or the upper layer of the current block. The default mode may be at least one of the Planar mode, the DC mode, or a predetermined directional mode. The predetermined directional mode may include at least one of the horizontal mode (mode V), the vertical mode (mode H), (mode V-k), (mode V+k), (mode H-k), or (mode H+k).

[0144] The MPM index can specify an MPM that is the same as the intra prediction mode of the current block among the MPMs in the MPM list. That is, the MPM specified by the MPM index may be set to the intra prediction mode of the current block.

[0145] Any one of Examples 1 to 4 can be selectively used to determine the intra prediction mode of the current block. Also, the intra prediction mode of the current block can be determined by a combination of at least two of Examples 1 to 4. A predetermined flag can be used for the selection. In this case, the flag can be encoded by an encoding device and transmitted as a signal.

[0146] Referring to FIG. 5, a reference sample used for intra prediction of the current block can be determined (S510).

[0147] The reference sample may be derived from an adjacent region (peripheral region) of the current block. The adjacent region of the current block may include at least one of the left side, the right side, the upper layer, the lower left layer, the upper left layer, the lower right layer, or the upper right layer of the current block.

[0148] The adjacent area may include one or more sample lines. Specifically, the number of sample lines belonging to the adjacent area is k, where k may be 1, 2, 3, 4, or a natural number greater than these. The value of k may be a fixed value pre-agreed upon in the encoding / decoding device, or may be variably determined based on the encoding information. For example, when the current block is of the first size (e.g., 4×4, 4×8, 8×4), the adjacent area may be arranged with one sample line. When the current block is of the second size (e.g., 8×8, 16×16, etc.), the adjacent area may be arranged with two sample lines. Based on the position of the adjacent area, the sample lines can be determined in the vertical or horizontal direction. Also, the sample lines may be in contact with the current block, or may be separated by a predetermined distance in the vertical and / or horizontal directions with respect to the current block.

[0149] The plurality of sample lines may be continuously present in the vertical and / or horizontal directions with respect to the current block or may be separated from each other by a predetermined distance. As an example, when two sample lines exist in the upper layer of the current block, the lowermost sample line among the two lines is named the first sample line and the second sample line, respectively, in the upward direction. In this case, the first sample line and the second sample line may be in contact with each other or may be separated by a predetermined distance. Here, the predetermined distance may be represented by i line lengths (i.e., width or height). Here, i may be 0, 1, 2, 3, or a natural number greater than that. As an example, when three sample lines exist in the upper layer of the current block, the lowermost sample line among the plurality of lines is named the first sample line, the second sample line, and the third sample line, respectively, in the upward direction. In this case, the first sample line and the second sample line may be in contact with each other, and the second sample line and the third sample line may be in contact with each other. Alternatively, the first to third sample lines may be separated by the predetermined distance. In this case, the interval (d1) between the first sample line and the second sample line may be the same as the interval (d2) between the second sample line and the third sample line. Alternatively, d1 may be set to be greater than d2. Conversely, d1 may be set to be less than d2. As an example, when four or more sample lines exist in the upper layer of the current block, the four sample lines can be determined in the same manner as in the case of the three sample lines. Note that this example is applicable not only to the sample lines located in the upper layer but also to the sample lines located on the left side. Here, specific descriptions are omitted.

[0150] The reference sample can be derived by using all or part of the samples belonging to the adjacent region.

[0151] (Example 1) Some samples in the adjacent area may be samples at positions pre-agreed by the encoding / decoding device. The pre-agreed positions may include at least one of the leftmost sample, the rightmost sample, or the middle sample of the upper sample line. The pre-agreed positions may include at least one of the uppermost sample, the lowermost sample, or the middle sample of the left sample line. Or, the pre-agreed positions may include at least one of the samples at odd-numbered positions in the upper and / or left sample lines, or at least one of the samples at odd-numbered positions. Or, the pre-agreed positions may include samples having x-coordinates that are multiples of j among the samples in the upper sample line, or samples having y-coordinates that are multiples of j among the samples in the left sample line. Here, j may be a natural number of 2, 3, 4, or greater.

[0152] (Example 2) Based on the encoded information, some samples in the adjacent area can also be variably determined. Here, the encoded information is as described above. Here, detailed description is omitted.

[0153] By selectively using any one of the above Example 1 or 2 or using a combination of Example 1 and 2, some samples can be specified. In this case, as described above, similarly, the interval between some samples can be set, but it is not limited thereto. The intervals between some samples can also be set to be different.

[0154] The number of the several samples may be one, two, three, four, or more, which is predefined in the encoding / decoding device. Note that, for the left adjacent region and the upper adjacent region of the current block, the number of the several samples can be defined differently respectively. For example, when the width of the current block is larger than the height, the number of the several samples (numSamA) belonging to the upper adjacent region may be larger than the number of the several samples (numSamL) belonging to the left adjacent region. Conversely, when the width of the current block is smaller than the height, numSamA may be smaller than numSamL. Alternatively, based on the encoding information, the number of the several samples can also be variably determined.

[0155] The samples in the adjacent region may be prediction samples or reconstructed samples. The prediction samples can be obtained by intra prediction or inter prediction. The reconstructed samples may be the reconstructed samples before applying the loop filter, or may be the reconstructed samples after applying the loop filter.

[0156] Note that the reference samples may be directly derived from the samples in the adjacent region (CASE 1), or may be derived in a way of downsampling the samples in the adjacent region (CASE 2). Either one of CASE 1 and CASE 2 can be selectively used. The selection can be made based on the encoding information. For example, when the size of the current block is less than a predetermined threshold, the reference samples can be derived based on CASE 1. Otherwise, the reference samples can be derived based on CASE 2. Here, the size may be represented by any one of the width of the current block, the height, the maximum value / minimum value of the width and the height, the ratio of the width and the height, or the product of the width and the height. As an example, when the current block is less than 8×8, the reference samples can be derived from the samples in the adjacent region. Otherwise, the reference samples can be derived by downsampling the samples in the adjacent region. The downsampling method will be further described with reference to FIGS. 6 and 7.

[0157] Referring to FIG. 5, a matrix used for intra prediction based on a matrix can be determined (S520).

[0158] The matrix can be determined by at least one of the intra prediction mode determined in step S500 or the size of the current block. Alternatively, the matrix can be determined by restricting to consider only the intra prediction mode of the current block. Alternatively, the matrix can be determined by restricting to consider only the size of the current block. The size may be represented by any one of the width or height, the minimum / maximum value of the width and height, the sum of the width and height, the number of samples belonging to the current block, etc. However, it is not limited thereto, and the matrix can be determined by further considering the coding information related to the current block. Here, the coding information is as described above. Here, detailed description is omitted.

[0159] Specifically, the matrix pre-agreed in the encoding / decoding device may be divided into a plurality of matrix groups. The plurality of matrix groups may be composed of a first matrix group, a second matrix group,..., an m-th matrix group. Here, m may be a natural number of 2, 3, 4, 5 or more. Based on the size of the current block, the current block can selectively use any one of the plurality of matrix groups. For example, when the size of the current block is 4×4, the first matrix group can be used. When the size of the current block is 8×4, 4×8, and 8×8, the second matrix group can be used. In addition, in other cases, the third matrix group can be used. The matrix group selected based on the size of the current block may include one or more matrix candidates. Any one of the plurality of matrix candidates can be determined by the matrix of the current block. The determination can be made based on the coding information (e.g., intra prediction mode) of the current block.

[0160] The number of the pre-agreed matrices may be the same as the number of the pre-defined intra prediction modes. Note that the number of the pre-agreed matrices may be less than the number of the pre-defined intra prediction modes. In this case, one matrix can match a plurality of intra prediction modes. For example, one matrix can match two intra prediction modes. In this case, the number of the pre-agreed matrices may be a value that is 1 / 2 times the number of the pre-defined intra prediction modes. However, it is not limited thereto, and the number of intra prediction modes matched with one matrix may be three, four, five, six or more.

[0161] As an example, the matching can be determined by considering the directionality and / or symmetry of the intra prediction mode.

[0162] The pre-defined intra prediction modes may include a directivity mode having a predetermined angle. The directivity modes may be divided into a first mode group having a horizontal directivity and a second mode group having a vertical directivity. If the number of directivity modes is 65, the first mode group may be set to include modes belonging to index 2 to index 34, and the second mode group may be set to include modes belonging to index 34 to index 66.

[0163] The encoding / decoding device defines only matrices for the first mode group. Similarly, the second mode group can use the matrices defined for the first mode group. Conversely, the encoding / decoding device defines only matrices for the second mode group. Similarly, the first mode group can use the matrices defined for the second mode group. In this case, the number of the pre-agreed matrices may be a value that is 1 / 2 times the number of the pre-defined intra prediction modes. As an example, when the number of mode groups having the symmetry is x, the number of the pre-agreed matrices may be a value that is 1 / x times the number of the pre-defined intra prediction modes. Here, x may be 3, 4 or more.

[0164] The symmetry may include symmetry of a prediction angle between a mode having a vertical directionality and a mode having a horizontal directionality, based on an intra prediction mode having an angle of -45°. Here, the intra prediction mode having a directionality has a prediction angle (PredAngle) according to each directionality. Here, the mode having a vertical directionality may include the mode having an angle of -45° and a mode having an angle of -45° < (PredAngle) - 45° along the x-axis direction from the mode, based on the intra prediction mode having the angle of -45°. Here, the mode having a horizontal directionality may include a mode having an angle of -45° < (PredAngle) - 45° along the y-axis direction from the mode other than the mode, based on the intra prediction mode having the angle of -45°.

[0165] Referring to FIG. 5, based on a reference sample and a matrix, a current block can be predicted (S530).

[0166] When determining a reference sample in step S510 and determining a matrix in step S520, the encoding / decoding apparatus can predict a current block based on the reference sample and the matrix.

[0167] The step of predicting the current block may include the step of applying the matrix to the reference sample to obtain a predicted sample of the DS block (hereinafter referred to as the first predicted sample). The DS block can represent the current block or can also represent the downsampled current block. That is, the DS block may have the same size as the current block. The size of the current block may be 1 / 2, 1 / 4, 1 / 8, or 1 / 16 of (at least one of the width or height). For example, when the current block is a 4×4, 4×8, or 8×4 block, the DS block may be a 4×4 block. Or, when the current block is an 8×8, 8×16, or 16×8 block, the DS block may be a 4×4 or 8×8 block. Or, when the current block is 16×16 or larger, the DS block may be an 8×8 or 16×16 block. However, the DS block is not limited to a square block and may be a non-square block. Or, the DS block may be limited to a square block. Here, the application of the matrix may include multiplying the reference sample by the weight value obtained from the matrix.

[0168] The step of obtaining the first predicted sample may include at least one of the step of adding an offset value or the step of filtering.

[0169] The step of obtaining the first predicted sample may further include the step of rearranging the first predicted sample. The rearrangement may be performed only when one matrix matches a plurality of intra prediction modes.

[0170] Or, when the intra prediction mode of the current block belongs to the first mode group having a horizontal directionality, the rearrangement can be performed. For example, when the intra prediction mode of the current block belongs to the first mode group having a horizontal directionality, the first predicted sample of the DS block is rearranged. When the intra prediction mode of the current block belongs to the second mode group having a vertical directionality, the rearrangement of the first predicted sample of the DS block may not be performed.

[0171] Conversely, when the intra prediction mode of the current block belongs to the first mode group having a vertical direction, the rearrangement can be performed. For example, when the intra prediction mode of the current block belongs to the first mode group having a horizontal direction, the rearrangement of the first prediction samples of the DS block is not performed. When the intra prediction mode of the current block belongs to the second mode group having a vertical direction, the rearrangement of the first prediction samples of the DS block can be performed.

[0172] As shown in Equation 1 below, the rearrangement can be performed. Here, x can represent the x-axis coordinate value, and y can represent the y-axis coordinate value. That is, the rearrangement can represent the process of assigning the first prediction samples of the (x, y) coordinates to the (y, x) coordinates.

[0173] [Equation 1] First prediction sample[x][y] = First prediction sample[y][x] Alternatively, the rearrangement of the present invention can also represent the process of rotating the DS block composed of the first prediction samples by a predetermined angle. Here, the predetermined angle can represent 90 degrees or 180 degrees in the clockwise direction, and can also represent 90 degrees or 180 degrees in the counterclockwise direction.

[0174] The step of predicting the current block may further include the step of performing upsampling on the current block based on at least one of the adjacent reconstructed samples or the first prediction samples to obtain a second prediction sample.

[0175] In the upsampling process, at least one of whether to perform the upsampling or how to perform the upsampling can be determined based on the encoded information of the current block. For example, at least one of whether to perform the upsampling or how to perform the upsampling can be determined based on the size of the DS block composed of the first prediction samples and the size of the current block. The size of the block can be represented by any one of the width or height, the minimum value / maximum value of the width and height, the sum of the width and height, the number of samples belonging to the block, and the like.

[0176] Only when the size of the DS block composed of the first prediction samples is smaller than the size of the current block, it is possible to determine whether to perform the upsampling.

[0177] The method of performing upsampling may include the step of allocating the first prediction samples to predetermined positions within the current block by using the ratio of the size of the DS block composed of the first prediction samples to the size of the current block, and the step of performing interpolation on the surplus area within the block. The surplus area can represent an area other than the area to which the first prediction samples are allocated in the current block. With reference to FIGS. 8 to 10, the method of allocating the first prediction samples and the method of interpolating the surplus area will be described in detail.

[0178] FIG. 6 is a diagram showing a method of determining reference samples by performing downsampling on adjacent blocks according to an embodiment to which the present invention can be applied.

[0179] Referring to FIG. 6, (a) in FIG. 6 shows a case where the adjacent regions used for intra prediction are located on the left side and the upper layer of the current block. As an example, the sample line located on the left side of the current block is in contact with the current block and consists of one sample line in the vertical direction. The sample line located on the upper layer of the current block is in contact with the current block and consists of one sample line in the horizontal direction.

[0180] The reference sample may include a downsampling region formed by performing downsampling on an adjacent region of the current block.

[0181] The downsampling can be performed from the average value, maximum value, minimum value, mode, or filtered value of all or some of the samples belonging to the adjacent region.

[0182] When derived from the average value, the downsampling region can be formed by a method of assigning the average values of different N samples to the samples in the downsampling region.

[0183] The different N samples may be samples arranged continuously or at a predetermined interval. The predetermined interval is an interval of the size of one or more samples. When there are a plurality of intervals, the plurality of intervals may be uniform or non-uniform. (Here, N is greater than 2 and less than the total number of samples belonging to the adjacent region.) Incidentally, the combination of the different N samples is called a sample group. In this case, the first sample group and the second sample group may or may not overlap.

[0184] As an example, FIG. 6 shows a case where N is 2, two sample groups do not overlap each other, the average values of two samples belonging to each sample group are assigned to one sample in the downsampling region, and downsampling is performed.

[0185] Alternatively, three consecutive samples (S1, S2, S3) can form a first sample group, and the average value of the three samples belonging to the first sample group can be assigned to a sample (DS1) in the downsampling region. Three consecutive samples (S2, S3, S4) can form a second sample group, and the average value of the three samples belonging to the second sample group can be assigned to a sample (DS2) in the downsampling region.

[0186] Alternatively, after determining the minimum or maximum value of two samples (S1, S2) belonging to the first sample group, it can be assigned to a sample (DS1) in the downsampling region. Similarly, after determining the minimum or maximum value of two samples (S3, S4) belonging to the second sample group, it can be assigned to a sample (DS2) in the downsampling region. This can also be similarly applied when the first / second sample group consists of three samples.

[0187] Alternatively, in the upper adjacent region, a sample at a predefined position among a plurality of samples belonging to the first sample group may be assigned to a sample (DS1) in the downsampling region. A sample at a predefined position among a plurality of samples belonging to the second sample group may be assigned to a sample (DS2) in the downsampling region. The predefined position can represent a fixed position predefined in the encoding / decoding device. As an example, it may be any one of the leftmost, rightmost, or middle positions. In the left adjacent region, samples at predefined positions among a plurality of samples belonging to each sample group may also be respectively assigned to samples in the downsampling region. In this case, the predefined position may be any one of the uppermost, lowermost, or middle positions.

[0188] FIG. 7 is a diagram showing a downsampling method based on a weighted average value according to an embodiment to which the present invention can be applied.

[0189] In this embodiment, the average value may be calculated by the following formula (hereinafter referred to as the first average expression).

[0190] Sum of samples belonging to the sample group / Number of samples Or, it may be calculated by the following formula (hereinafter referred to as the second average expression) Sum(Weight value × Samples belonging to the sample group) / Number of samples (a) in FIG. 7 shows the case where the sample group is composed of three samples. In this case, the weight values applied to the three samples may be determined as a ratio of 1:2:1. As shown in (b) in FIG. 7, when the sample group is composed of five samples, the weight values may be determined as a ratio of 1:1:4:1:1. As shown in (c) in FIG. 7, when the sample group is composed of six samples, the weight values may be determined as a ratio of 1:2:1:2:2:1 or 1:2:2:1:2:1 in the Z direction starting from the upper left layer. Note that (a) and (c) in FIG. 7 show the weight values applied to the upper layer adjacent region, which is similarly applicable to the left adjacent region.

[0191] The average value may include a result value derived by applying a predetermined operation to a plurality of average values calculated by the first average expression or the second average expression. Here, the predetermined operation may be the first average expression or the second average expression. For example, when three samples (i.e., the first to third samples) belong to the sample group, the average value (the first value) of the first sample and the second sample and the average value (the second value) of the second sample and the third sample can be calculated respectively. The average value can be derived from the average value of the calculated first value and second value.

[0192] The downsampling method may be applied only to the upper adjacent region. Conversely, it may be applied only to the left adjacent region. Or, the downsampling method according to FIG. 6 (hereinafter referred to as the first method) can be applied to any one of the upper or left adjacent regions, and the downsampling method according to FIG. 7 (hereinafter referred to as the second method) can be applied to the other one of the upper or left adjacent regions.

[0193] Note that considering the size / shape of the current block, at least one of the first method or the second method can be selectively used. For example, when the width of the current block is larger than a predetermined threshold, the first method is applicable to the upper adjacent region of the current block, and otherwise, the second method is applicable. In a similar manner, downsampling can also be performed on the height of the current block. Or, when the current block is non-square, the first method can be applied to any one of the upper or left adjacent regions, and the second method can be applied to the other region. In this case, when the width of the current block is larger than the height, the first method is applicable to the upper adjacent region, and the second method is applicable to the left adjacent region. Conversely, when the width of the current block is smaller than the height, the second method is applicable to the upper adjacent region, and the first method is applicable to the left adjacent region. When the current block is square, the same downsampling method can be used in the upper and left adjacent regions. Here, the downsampling method may be limited to the first method.

[0194] FIG. 8 is a diagram showing a method of dividing the first prediction sample and interpolating other regions according to an embodiment to which the present invention can be applied.

[0195] Referring to (a) in FIG. 8, the prediction sample of the DS block may be assigned to the prediction sample at a predetermined position in the current block. Here, the predetermined position can be determined by considering the ratio of the sizes of the current block and the DS block. For example, the correspondence relationship between the prediction samples of the DS block and the current block may be defined by Equation 2.

[0196] [Formula 2] The first prediction sample curBLK[(x + 1)×r - 1][(y + 1)×r - 1] = the first prediction sample dsBLK[x][y] Here, r represents the ratio of the size of the current block to the DS block, and x and y are the x-axis and y-axis coordinates of the first prediction sample within each DS block, respectively. The first prediction sample curBLK can represent the position of the first prediction sample within the current block, and the first prediction sample dsBLK can represent the position of the first prediction sample within the DS block.

[0197] Regarding the interpolation, as shown in (b) in FIG. 8, at least one of the first prediction sample assigned to the current block or the reconstructed sample adjacent to the current block (hereinafter referred to as the interpolation reference sample) is used to derive a sample to which no assignment has been made to the first prediction sample in the current block (hereinafter referred to as the interpolation symmetric sample). Note that the interpolation reference sample may further include a prediction sample generated by interpolation before the current interpolation target sample (i.e., the immediately preceding interpolation target sample).

[0198] The position and range of the reconstructed sample adjacent to the current block are the same as those of the reference sample. Here, detailed description is omitted.

[0199] According to the position of the sample to be interpolated, the interpolation reference sample may be composed of a plurality of first prediction samples, or may be composed of at least one first prediction sample and at least one reconstructed adjacent sample. The reconstructed adjacent sample can selectively use any one of the x - coordinate or y - coordinate samples that are the same as the sample to be interpolated, or can use at least one of the x - coordinate or y - coordinate and a plurality of samples of the sample to be interpolated. The selection can be made based on the position of the sample to be interpolated. For example, when the sample to be interpolated has the same x - coordinate as the first prediction sample, the reconstructed adjacent sample may include only the samples having the same x - coordinate as the sample to be interpolated. Conversely, when the sample to be interpolated has the same y - coordinate as the first prediction sample, the reconstructed adjacent sample may include only the samples having the same y - coordinate as the sample to be interpolated. Or, the reconstructed adjacent sample may include a plurality of samples located on the same horizontal and vertical lines as the sample to be interpolated.

[0200] The sample to be interpolated may be derived from the representative value of a plurality of interpolation reference samples. Here, the representative value may include any one of an average value, a minimum value, a maximum value, a mode value, or a median value.

[0201] The average value may be calculated by the following formula (hereinafter referred to as the first average expression).

[0202] Sum of difference - value reference samples / Number of difference - value reference samples Or, it may be calculated by the following formula (hereinafter referred to as the second average expression).

[0203] Sum(Weight value × Difference - value reference sample) / Number of difference - value reference samples Based on the relative / absolute distance between the sample to be interpolated and the interpolation reference sample, the weight value according to the second average expression can be determined. This will be described in detail with reference to FIG. 9.

[0204] FIG. 9 is a diagram showing the assignment of weight values to distances in an interpolation step according to an embodiment to which the present invention can be applied.

[0205] The weight value of the present invention may include a weight value determined based on the distance from the interpolation target sample to the interpolation reference sample. As an example, referring to FIG. 9, when interpolation is performed on the first interpolation target sample 910, since the ratio of the distance from the first interpolation target sample 910 to the first interpolation reference sample 911 and the distance from the first interpolation target sample 910 to the second interpolation reference sample 912 is 3:1, the ratio of the weight values applied to the first interpolation reference sample 911 and the second interpolation reference sample may be 1:3. When interpolation is performed on the second interpolation target sample 920, since the ratio of the distance from the second interpolation target sample 920 to each first interpolation reference sample 921 and the distance from the second interpolation target sample 920 to the second interpolation reference sample 922 is 1:1, the ratio of the weight values applied to the first interpolation reference sample and the second interpolation reference samples 921, 922 may be 1:1.

[0206] Note that the interpolation filter of the present invention may have a direction. The direction may include directions such as vertical, horizontal, z-shaped, diagonal, etc.

[0207] The interpolation can be performed based on a predetermined wired order. The priority may be either performing interpolation in the vertical direction first and then in the horizontal direction (first order) or performing interpolation in the horizontal direction first and then in the vertical direction (second order). Or, it may be performing interpolation simultaneously in the vertical and horizontal directions (third order).

[0208] Interpolation can be performed on the current block using only any one of the first to third orders. Or, interpolation can be performed using a combination of at least two of the first to third orders. Details of the interpolation order can be understood with reference to FIG. 10.

[0209] FIG. 10 is a diagram showing the order of interpolation steps according to an embodiment to which the present invention can be applied.

[0210] (a) in FIG. 10 is the first order in FIG. 9. Specifically, first, interpolation is performed on the vertical line to which the first prediction sample belongs, and then, based on the interpolated line and the interpolation reference sample on the left side of the current block, interpolation can be performed on the horizontal line.

[0211] (b) in FIG. 10 is the second order in FIG. 9. Specifically, first, interpolation is performed on the horizontal line to which the first prediction sample belongs, and then, based on the interpolated line and the interpolation reference sample in the upper layer of the current block, interpolation can be performed on the vertical line.

[0212] (c) in FIG. 10 is the third order in FIG. 9. First, interpolation is performed on the vertical and horizontal lines to which the first prediction sample belongs. Subsequently, interpolation is performed on the remaining samples that have not been interpolated. In this case, interpolation can be performed on only the vertical line or the horizontal line, or interpolation can be performed on the vertical and horizontal lines simultaneously. When interpolation is performed on the vertical and horizontal lines simultaneously, one interpolation target sample may have the first interpolation value on the vertical line and the second interpolation value on the horizontal line at the same time. In this case, a representative value of the first interpolation value and the second interpolation value can be assigned to the interpolation target sample. Here, the representative value can be derived from the average value, minimum value, maximum value, mode value, or median value.

[0213] The interpolation order may be a pre-agreed order in the encoding / decoding device, or may be selectively determined based on the encoding information of the current block. Here, the encoding information is as described above. Therefore, a detailed description is omitted here.

[0214] The order can be determined based on the size of the block. The size of the block may be represented by either the width or the height, the minimum value / maximum value of the width and the height, the sum of the width and the height, the number of samples belonging to the block, and the like.

[0215] For example, when the size of the current block is larger than a predetermined threshold value, the first interpolation can be performed. Otherwise, the second interpolation can be performed. Conversely, when the size of the current block is less than the predetermined threshold value, the second interpolation can be performed, otherwise, the first interpolation can be performed. The threshold value may be 8, 16, 32 or a natural number greater than that.

Claims

1. 1. A video signal processing method, performed by a decoder, comprising: determining an intra-prediction mode for a current block; determining a reference sample to be used for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode; predicting the current block based on the reference samples and the matrix; predicting the current block based on the reference sample and the matrix, generating a prediction block by applying the matrix to the reference samples, wherein samples of the prediction block are generated at predetermined positions within the current block; The matrix is ​​determined based on coding information of the current block; predicting the current block based on the reference sample and the matrix, The video signal processing method further comprises the step of transposing all or a part of the prediction samples of the generated prediction block.

2. The step of determining the reference sample comprises: determining a neighboring region of the current block; downsampling the determined adjacent region; The contiguous region is divided into a plurality of sample groups; the sample group consists of one or more samples; a representative value of the sample group is determined from the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value.

2. The video signal processing method of claim 1.

3. The encoding information includes the size and shape of the current block.

2. The video signal processing method of claim 1.

4. predicting the current block based on the reference sample and the matrix, performing interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block.

2. The video signal processing method of claim 1.

5. 1. A video signal processing method, performed by an encoder, comprising: determining an intra-prediction mode for a current block; determining a reference sample to be used for intra prediction of the current block; determining a predetermined matrix based on the intra prediction mode; predicting the current block based on the reference samples and the matrix; predicting the current block based on the reference sample and the matrix, generating a prediction block by applying the matrix to the reference samples, wherein samples of the prediction block are generated at predetermined positions within the current block; The matrix is ​​determined based on coding information of the current block; predicting the current block based on the reference sample and the matrix, The video signal processing method further comprises the step of transposing all or a part of the prediction samples of the generated prediction block.

6. The step of determining the reference sample comprises: determining a neighboring region of the current block; downsampling the determined adjacent region; The contiguous region is divided into a plurality of sample groups; the sample group consists of one or more samples; a representative value of the sample group is determined from the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value.

6. A video signal processing method according to claim 5.

7. The encoding information includes the size and shape of the current block.

6. A video signal processing method according to claim 5.

8. predicting the current block based on the reference sample and the matrix, performing interpolation for the current block based on at least one of the predicted block or reconstructed samples adjacent to the current block.

6. A video signal processing method according to claim 5.

9. A decoding device for processing a video signal, comprising: a first determining unit configured to determine an intra-prediction mode of a current block; a second determination unit configured to determine a reference sample to be used for intra prediction of the current block; a third determination unit configured to determine a predetermined matrix based on the intra prediction mode; a generation unit configured to generate a prediction block by applying the matrix to the reference samples, wherein samples of the prediction block are generated at predetermined positions within the current block; The matrix is ​​determined based on coding information of the current block; The generating unit further comprises: A decoding device for processing a video signal, configured to transpose all or part of the prediction samples of said generated prediction block.

10. The second determination unit further comprises: determining a neighboring region of the current block; configured to perform downsampling on the determined adjacent region; The contiguous region is divided into a plurality of sample groups; the sample group consists of one or more samples; a representative value of the sample group is determined from the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value. Decoding device for processing video signals according to claim 9.

11. The encoding information includes the size and shape of the current block. Decoding device for processing video signals according to claim 9.

12. The generating unit further comprises: configured to perform interpolation for the current block based on at least one of the predicted block or reconstructed samples neighboring the current block. Decoding device for processing video signals according to claim 9.

13. 1. An encoding device for video signal processing, comprising: a first determining unit configured to determine an intra-prediction mode of a current block; a second determination unit configured to determine a reference sample to be used for intra prediction of the current block; a third determination unit configured to determine a predetermined matrix based on the intra prediction mode; a generation unit configured to generate a prediction block by applying the matrix to the reference samples, wherein samples of the prediction block are generated at predetermined positions within the current block; The matrix is ​​determined based on coding information of the current block; The generating unit further comprises: An encoding device for processing a video signal, configured to transpose all or some prediction samples of said generated prediction block.

14. The second determination unit further comprises: determining a neighboring region of the current block; configured to perform downsampling on the determined adjacent region; The contiguous region is divided into a plurality of sample groups; the sample group consists of one or more samples; a representative value of the sample group is determined from the reference sample; The representative value is any one of an average value, a minimum value, a maximum value, a mode value, and a median value. Encoding device for processing video signals according to claim 13.

15. The encoding information includes the size and shape of the current block. Encoding device for processing video signals according to claim 13.

16. The generating unit further comprises: configured to perform interpolation for the current block based on at least one of the predicted block or reconstructed samples neighboring the current block. Encoding device for processing video signals according to claim 13.

17. A computer-readable storage medium having a computer program and a bitstream stored thereon, A computer-readable storage medium, the computer program causing a processor to perform the video signal processing method of any one of claims 1 to 4, so as to decode the bitstream to generate video.

18. A computer-readable storage medium having a computer program and a bitstream stored thereon, A computer-readable storage medium, the computer program causing a processor to execute the video signal processing method according to any one of claims 5 to 8, so as to generate the bitstream.

Images