Intra prediction device, encoding apparatus, decoding apparatus and program

Abstract

To improve encoding efficiency in a case of performing directivity prediction.SOLUTION: An intra prediction device for performing intra prediction in a block unit constituting an image comprises: an area division unit 1722a which divides an object block to which an intra prediction mode having a linear prediction direction is allocated into a plurality of sub areas; a derivation unit 1722b which derives a sub prediction mode for each of the plurality of sub areas on the basis of a reference curve obtained according to the intra prediction mode, the block shape of the object block and the division number of the object block; and an area prediction unit 1722c which generates a plurality of prediction areas by predicting each of the plurality of sub areas using the sub prediction mode; and an area composition unit 1722d which outputs a prediction block with respect to the object block by combining the plurality of prediction areas.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an intra prediction device, an encoding device, a decoding device, and a program. [Background technology]

[0002] In order to transmit a video signal containing a huge amount of information within a limited bandwidth, it is necessary to perform coding. In the conventional video coding method and the next-generation video coding method (VVC), which have been standardized, a frame is divided into small blocks (hereafter referred to as coding blocks), and prediction and coding are performed for each block.

[0003] Prediction methods can be broadly divided into two: intra prediction and inter prediction. Intra prediction is a method of making predictions that utilize spatial correlation using decoded blocks adjacent to the block to be coded within a frame. Inter prediction is a method of making predictions that utilize temporal correlation using decoded frames other than the frame to which the block to be coded belongs. In general, frames that are close in time in video are similar to each other, so inter prediction has higher prediction accuracy than intra prediction, and the amount of information in the residual signal (prediction residual) transmitted to the decoding side is small.

[0004] On the other hand, intra-prediction is more difficult to achieve with high accuracy than inter-prediction, so the amount of information in the residual signal transmitted to the decoding side is overwhelmingly greater than that of inter-prediction. Transmitting the residual signal using intra-prediction within a limited bandwidth requires coarser quantization, which makes it more likely that blocks coded and decoded using intra-prediction will be degraded. Images decoded using intra-prediction are referenced during inter-prediction, so any degradation that occurs during intra-prediction also affects inter-predicted frames that reference this intra-predicted frame, resulting in degradation of the overall image. For these reasons, research is underway into coding methods that can compress intra-predicted frames with high efficiency.

[0005] VVC intra prediction introduces planar prediction, which performs curved prediction using decoded reference pixels (hereinafter simply referred to as "reference pixels") adjacent to the block to be coded, DC prediction, which performs prediction based on the average value of the reference pixels, and directional prediction, which performs prediction by linearly copying some of the reference pixels. In VVC directional prediction, the number of prediction directions has increased to 65, and 65 prediction modes are specified. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] JVET-S2001 Versatile Video Coding (Draft 10) Summary of the Invention [Problem to be solved by the invention]

[0007] Although the prediction accuracy of VVC's directional prediction has improved by increasing the number of prediction directions to 65, prediction is still performed linearly in all directional prediction modes. However, in general natural images, object boundaries are often curved, and linear prediction cannot predict these object boundaries with high accuracy, resulting in a problem of reduced coding efficiency.

[0008] In order to predict curved object boundaries with high accuracy, it is necessary to finely divide the block to be coded at the object boundary and apply the most accurate directional prediction to each of these fine blocks. However, as the block division becomes finer, the amount of flag information related to the block division increases, which may result in a decrease in coding efficiency.

[0009] Therefore, an object of the present invention is to provide an intra prediction device, an encoding device, a decoding device, and a program that are capable of improving the encoding efficiency when directional prediction is performed. [Means for solving the problem]

[0010] The intra prediction device according to the first aspect is an intra prediction device that performs intra prediction on a block-by-block basis that constitutes an image, and includes: a region division unit that divides a target block to which an intra prediction mode having a linear prediction direction is assigned into a plurality of sub-regions; a derivation unit that derives a sub-prediction mode for each of the plurality of sub-regions based on a reference curve obtained according to the intra prediction mode, the block shape of the target block, and the number of divisions of the target block; a region prediction unit that predicts each of the plurality of sub-regions using the sub-prediction mode to generate a plurality of prediction regions; and a region synthesis unit that synthesizes the plurality of prediction regions and outputs a prediction block for the target block.

[0011] The encoding device according to a second aspect is summarized as including the intra prediction device according to the first aspect.

[0012] A decoding device according to a third aspect is summarized as including the intra prediction device according to the first aspect.

[0013] A program according to a fourth aspect causes a computer to function as the intra prediction device according to the first aspect. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide an intra prediction device, an encoding device, a decoding device, and a program that are capable of improving the encoding efficiency when directional prediction is performed. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Figure 2] FIG. 10 is a diagram illustrating the operation of an intra prediction unit according to the embodiment. [Figure 3] FIG. 10 is a diagram showing intra prediction modes of an intra prediction unit according to an embodiment. [Figure 4] FIG. 2 is a diagram illustrating a configuration of an intra curve prediction unit of the encoding device according to the embodiment. [Figure 5] 10A and 10B are diagrams illustrating an operation of an area dividing unit according to the embodiment. [Figure 6] FIG. 10 is a diagram illustrating a sub-prediction mode derivation operation according to the embodiment. [Figure 7] FIG. 1 is a diagram illustrating a first method according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating a second method according to the embodiment. [Figure 9] FIG. 10 is a diagram illustrating another technique according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating another technique according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating another technique according to an embodiment. [Figure 12] FIG. 1 is a diagram illustrating a configuration of a decoding device according to an embodiment. [Figure 13] FIG. 2 is a diagram illustrating a configuration of an intra curve prediction unit of a decoding device according to an embodiment. [Figure 14] FIG. 2 is a diagram illustrating an operation of a decoding device according to an embodiment. [Figure 15] FIG. 10 is a diagram illustrating an operation of an intra curve prediction unit according to the embodiment. [Figure 16] 10A and 10B are diagrams illustrating examples of prediction by an intra curve prediction unit according to the embodiment. [Figure 17] 10A and 10B are diagrams illustrating examples of prediction by an intra curve prediction unit according to the embodiment. [Figure 18] FIG. 10 shows another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. The encoding device and the decoding device according to the embodiment encode and decode moving images, respectively, as typified by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] <Encoder Configuration> First, the configuration of the encoding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an encoding device 1 according to this embodiment.

[0018] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a subtraction unit 110, a transformation and quantization unit 120, an entropy encoding unit 130, an inverse quantization and inverse transform unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0019] The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) that make up a moving image, into a plurality of image blocks, and outputs the image blocks obtained by division to the subtraction unit 110. The size of the image blocks is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image blocks is not limited to squares and may be rectangular (non-square).

[0020] An image block is a unit of coding performed by the encoding device 1 (i.e., a block to be coded) and a unit of decoding performed by the decoding device (i.e., a block to be decoded). Such an image block is sometimes called a CU (Coding Unit). When there is no particular distinction between a block to be coded and a block to be decoded, they are simply called a target block.

[0021] The block division unit 100 divides the luminance signal and the chrominance signal that constitute the image into blocks, and outputs luminance blocks and chrominance blocks. The division may be controlled independently for the luminance signal and the chrominance signal. When there is no particular distinction between the luminance block and the chrominance block, they are simply referred to as the block to be coded.

[0022] The subtraction unit 110 calculates a prediction residual representing the difference (error) between the current block to be coded output from the block division unit 100 and a prediction block obtained by predicting the current block to be coded by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the prediction block from each pixel value of the block, and outputs the calculated prediction residual to the transformation and quantization unit 120.

[0023] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.

[0024] The transform unit 121 performs a transform process on the prediction residual output from the subtraction unit 110 to calculate transform coefficients, and outputs the calculated transform coefficients to the quantization unit 122. The transform process refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), etc. The transform process may include a transform skip that adjusts the pixel block signal by scaling or the like without converting it into a frequency domain signal.

[0025] The quantization unit 122 quantizes the transform coefficients output from the transform unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized transform coefficients to the entropy coding unit 130 and the inverse quantization and inverse transform unit 140.

[0026] The entropy coding unit 130 performs entropy coding on the transform coefficients output from the quantization unit 122, compresses the data, generates a coded stream (bit stream), and outputs the coded stream to the outside of the coding device 1. For the entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used.

[0027] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.

[0028] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transform coefficients by inverse quantizing the transform coefficients output from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored transform coefficients to the inverse transform unit 142.

[0029] The inverse transform unit 142 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121. For example, if the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse transform processing on the transform coefficients output from the inverse quantization unit 141 to restore prediction residuals, and outputs restored prediction residuals, which are the restored prediction residuals, to the synthesis unit 150.

[0030] The synthesis unit 150 synthesizes, on a pixel-by-pixel basis, the reconstructed prediction residual output from the inverse transform unit 142 with the predicted block output from the prediction unit 170. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to decode (reconstruct) the block to be coded, and outputs the decoded block to the memory 160. Note that the decoded block may also be called a reconstructed block.

[0031] The memory 160 stores the decoded blocks output from the synthesis unit 150 and accumulates the decoded blocks as decoded images on a frame-by-frame basis. The memory 160 outputs the stored decoded blocks or decoded images to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160.

[0032] The prediction unit 170 performs prediction processing on a block-by-block basis. The prediction unit 170 has an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173. In addition to intra prediction (intra linear prediction) and inter prediction similar to conventional methods, the prediction unit 170 calculates the RD cost of intra curve prediction, determines the optimal prediction mode, and generates a predicted block in the optimal prediction mode.

[0033] The inter prediction unit 171 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 171 uses a decoded image stored in the memory 160 as a reference image to calculate a motion vector by a technique such as block matching, predicts a current block to be coded, generates an inter prediction block, and outputs the generated inter prediction block to the switching unit 173. Here, the inter prediction unit 171 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130.

[0034] The intra prediction unit 172 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 172 generates an intra prediction block by referring to reference pixels, which are decoded pixels surrounding a block to be coded, among decoded images stored in the memory 160, and outputs the generated intra prediction block to the switching unit 173. In this embodiment, the intra prediction unit 172 corresponds to an intra prediction device on the coding side.

[0035] The intra prediction unit 172 has a prediction mode determination unit 1721 that determines an intra prediction mode predMode to be applied to a current block from among a plurality of intra prediction modes. The intra prediction unit 172 predicts the current block using the intra prediction mode predMode determined by the prediction mode determination unit 1721. The intra prediction unit 172 outputs flag information indicating the determined intra prediction mode predMode to the entropy coding unit 130. In this embodiment, the intra prediction mode predMode for each current block is signaled to the decoding side by the entropy coding unit 130, but the decoding side may also determine the intra prediction mode predMode based on parameters of the current block (position, block shape, and prediction mode of the neighboring region).

[0036] The intra prediction unit 172 according to this embodiment includes an intra curve prediction unit 1722. The intra curve prediction unit 1722 will be described in detail later.

[0037] The switching unit 173 switches between the inter-predicted block output from the inter-prediction unit 171 and the intra-predicted block output from the intra-prediction unit 172, and outputs either the predicted block to the subtraction unit 110 and the synthesis unit 150.

[0038] Next, the intra prediction unit 172 according to this embodiment will be described. Fig. 2 is a diagram for explaining the operation of the intra prediction unit 172 according to this embodiment.

[0039] As shown in Figure 2, the intra prediction unit 172 performs intra prediction using one of the following intra prediction modes: planar prediction, which performs curved prediction using decoded reference pixels adjacent to the block to be coded; DC prediction, which performs prediction using the average value of the reference pixels; and directional prediction, which performs prediction by linearly copying some of the reference pixels. The reference pixels are selected from the reference pixel column above and the reference pixel column to the left of the block to be coded, depending on the intra prediction mode. In this embodiment, 65 prediction directions for directional prediction, i.e., 65 intra prediction modes, are provided.

[0040] In the following, an example will be described in which the available reference pixels are decoded pixel signals adjacent to the upper side or left side of the block to be coded. However, when multi-column reference pixel prediction (Multi-reference line prediction) is introduced, which determines the reference pixel positions to be used for prediction processing among multiple reference pixel columns introduced in the VVC standard, prediction processing may be performed using reference pixels that are not adjacent to the block to be coded, for example, reference pixels at positions spaced one pixel apart from the block to be coded.

[0041] FIG. 3 is a diagram showing intra prediction modes of the intra prediction unit 172 according to this embodiment.

[0042] As shown in Fig. 3, there are 67 candidate intra prediction modes, ranging from 0 to 66. Intra prediction mode mode "0" is planar prediction, intra prediction mode mode "1" is DC prediction, and intra prediction modes "2" to "66" are directional prediction. In directional prediction, the direction of an arrow indicates the reference direction, the starting point of the arrow indicates the position of a pixel to be predicted, and the end point of the arrow indicates the position of a reference pixel used in predicting this pixel to be predicted.

[0043] The reference directions parallel to the diagonal line passing through the upper right and lower left vertices of the block to be coded are mode "2," which is an intra prediction mode that references the lower left direction, and mode "66," which is an intra prediction mode that references the upper right direction. Mode numbers are assigned clockwise from mode "2" to mode "66" at predetermined angle intervals. Mode numbers "-2" to "-14" and mode numbers "67" to "80" are only applied when the block shape of the block to be coded is rectangular (non-square). The following mainly describes an example in which the block shape of the block to be coded is square and mode numbers "34" to "50" are used.

[0044] Next, the intra curve prediction unit 1722 according to one embodiment will be described. As described above, general directional prediction performs prediction by linearly copying a portion of reference pixels. However, in general natural images, object boundaries are often curved, and linear prediction cannot predict these object boundaries with high accuracy, resulting in a problem of reduced coding efficiency.

[0045] The intra curve prediction unit 1722 according to this embodiment divides a current block to be coded, to which an intra prediction mode predMode having a linear prediction direction is assigned, into multiple sub-regions, derives a sub-prediction mode for each of the multiple sub-regions according to a reference curve obtained from the linear prediction direction, predicts each of the multiple sub-regions using the sub-prediction mode to generate multiple prediction regions, and combines the multiple prediction regions to output a predicted block for the current block to be coded. This enables highly accurate prediction of curved object boundaries, which is expected to improve coding efficiency.

[0046] In this embodiment, by performing intra curve prediction using a common algorithm on the encoding side and the decoding side, signaling of flag information indicating the sub-prediction mode is unnecessary, and the intra-prediction mode predMode for each block to be encoded can be signaled as in the past. Specifically, the entropy encoding unit 130 does not include flag information indicating the sub-prediction mode in the encoded stream, but includes flag information indicating the intra-prediction mode predMode in the encoded stream. This prevents an increase in the amount of flag information signaled to the decoding side and prevents a decrease in encoding efficiency due to the amount of flag information.

[0047] The entropy coding unit 130 may include, in the coded stream, flag information indicating whether or not intra prediction has been applied for each coding target block. The entropy coding unit 130 may include, in the coded stream, flag information indicating whether or not intra curve prediction has been applied for a coding target block to which intra prediction has been applied.

[0048] Alternatively, the prediction process may be controlled using an existing flag. For example, flag information indicating whether intra curve prediction is applicable may be shared with flag information (intra_subpartitions_mode_flag) indicating whether ISP mode is applicable in VVC, and flag information indicating whether the ISP mode or intra curve prediction is applied may be added only when the ISP mode is applicable (e.g., single_prediction_mode_flag, which indicates the conventional ISP mode when True and the intra curve prediction when False) to determine whether curved intra prediction is applied. In this case, flag information indicating the split direction of curved intra prediction may also be shared with flag information (intra_subpartitions_split_flag) indicating the split direction of the ISP mode.

[0049] Fig. 4 is a diagram showing the configuration of the intra curve prediction unit 1722 according to this embodiment. As shown in Fig. 4, the intra curve prediction unit 1722 includes a region division unit 1722a, a derivation unit 1722b, a region prediction unit 1722c, and a region synthesis unit 1722d.

[0050] 5 is a diagram showing the operation of the region dividing unit 1722a according to this embodiment. As shown in Fig. 5, the region dividing unit 1722a divides a current block to be coded, to which an intra prediction mode predMode having a linear prediction direction is assigned, into a plurality of sub-regions subBlock_i (i = 0, 1, ..., n).

[0051] In this embodiment, an example will be described in which the region dividing unit 1722a divides the encoding target block into four vertically, but the division direction and the number of divisions may be determined based on parameters of the encoding target block (position, block shape, and prediction mode) or parameters of adjacent decoded regions. For example, the encoding target block may be divided into a quadtree division into a square shape, or may be divided horizontally or vertically into a predetermined number of blocks. Furthermore, information on the division shape and position of each sub-region determined on the encoding side may be signaled from the entropy coding unit 130 to the decoding side. Furthermore, the division shape does not have to be rectangular, and division into triangular or trapezoidal regions introduced in VVC may also be used.

[0052] The derivation unit 1722b derives, for each of the sub-regions subBlock_i, a sub-prediction mode subPredMode according to a reference curve obtained from the linear prediction direction of the intra-prediction mode predMode, based on the intra-prediction mode predMode.

[0053] First, the derivation unit 1722b derives a sub-prediction mode subPredMode for each of the sub-regions subBlock_i. In this embodiment, the derivation unit 1722b sets the intersections of the boundary lines of each of the sub-regions subBlock_i and the reference curve as reference points, and derives a sub-prediction mode subPredMode for each of the sub-regions subBlock_i based on the straight lines obtained by connecting the reference points.

[0054] FIG. 6 is a diagram showing the operation of deriving the sub prediction mode subPredMode according to this embodiment.

[0055] As shown in FIG. 6(a), the derivation unit 1722b sets point C0=(0,0) and point C2=(w-1,h-1). Here, "w" represents the width of the block to be coded, and "h" represents the height of the block to be coded. The derivation unit 1722b also sets C1=(c x ,cy )

[0056] 6(b), the derivation unit 1722b uses a quadratic Bezier curve with these three points as control points as a reference curve, and uses the intersection of the reference curve and the division line as a reference point. In this embodiment, the reference curve is a quadratic Bezier curve, but other curves may be used, or prediction parameters corresponding to the intra prediction mode predMode and the division shape may be stored as a table and referenced.

[0057] The reference point is determined based on the prediction mode of the block to be coded and the block shape (for example, the width W and height H of the block to be coded) as follows: First, a control point C1 is obtained using intraPredAngle defined in VVC as follows.

[0058]

number

[0059] Next, the Bezier curve Bezier[t] can be calculated as follows:

[0060]

number

[0061] The reference point can be found by solving Bezier[t] for t, but a table may be stored in advance according to the prediction mode and block shape of the block to be coded. i Let's say.

[0062] As shown in Fig. 6(c), the derivation unit 1722b derives the sub-prediction mode subPredMode for each sub-region subBlock_i by selecting the intra prediction mode (see Fig. 3) having the slope closest to the slope of the line obtained by connecting the reference points. Note that although the slope of the line is the slope between two points here, the slope may also be obtained using a method such as differentiation of the curve at the reference points.

[0063] For example, the derivation unit 1722b calculates the slope [i] of the i-th sub-area subBlock_i as

[0064]

number

[0065] Calculate as follows, where w=W / 4.

[0066] In addition, a table slopeTable that associates the slope with the prediction mode is slopeTable = [0, 1, 2, 3, 4, 6, 6, 8, 8, 10, 10, 12, 12, 14, 14, 16, 16, 18, 18, 20, 20, 20, 23, 23, 23, 26, 26, 26, 29, 29, 29, 32, 32], then the prediction mode of the ith subregion, subBlock_i, is [i] is

[0067]

number

[0068] It can be calculated by:

[0069] Second, the derivation unit 1722b derives reference pixels for predicting the sub-region subBlock using the derived sub-prediction mode subPredMode.

[0070] The derivation unit 1722b calculates the reference pixel subRef of the sub-region subBlock_0 in the leftmost column. When deriving _0[x], the reference pixel is derived using the same method as the conventional method as follows.

[0071] subRef_0[ x ] = p[ -1 + x ][ -1 ] (0 ≦ x ≦ iw+1) subRef_0[ x ] = p[ -1][ -1 + Min( ( x * invAngle + 256 ) >> 9, h ) (-H ≦x ≦ -1 )

[0072] When deriving reference pixels (subBlock_i[x], i=1, 2, 3) of a sub-region subBlock_i other than the leftmost column, the derivation unit 1722b derives the reference pixels as follows.

[0073] subRef_i[x] = p[ iw+x-1 ][ -1 ] ( 0 ≦ x ≦ iw+1 )

[0074] Here, the reference pixels for x=-H...-1 are expanded in accordance with the sub-prediction modes subPredMode from subBlock_0 to subBlock_i-1. There are two methods for expanding the reference pixels:

[0075] First, method 1 will be described. Method 1 is a method that allows for highly accurate prediction but requires a long processing time. Specifically, in method 1, multiple sub-regions subBlock_i are predicted sequentially. A derivation unit 1722b derives predicted pixels included in the predicted sub-region subBlock_i as reference pixels. A region prediction unit 1722c uses the derived reference pixels to predict an unpredicted sub-region subBlock_i adjacent to the predicted sub-region subBlock_i.

[0076] FIG. 7 is a diagram showing method 1 according to this embodiment.

[0077] As shown in Figure 7, the derivation unit 1722b expands the reference pixels according to the sub-prediction mode subPredMode and the pixel values ​​of the predicted sub-region subBlock_i. This method expands the reference pixels of the adjacent sub-region subBlock_i using the pixel values ​​of the predicted sub-region subBlock_i, so high-precision prediction is expected. Furthermore, this method can be easily implemented using algorithms used in conventional coding methods.

[0078] subRef_i[x] is calculated as follows using the rightmost column predSample[iw-1][y] of predicted pixels in the sub-region subBlock_i-1 predicted by the region prediction unit 1722c.

[0079] subRef_i[x] = predSample[iw-1][ -1+ Min( ( x * invAngle + 256 ) >> 9, H )] (-H-1 ≦ x ≦ -2)

[0080] 7(a), the derivation unit 1722b derives reference pixels subRef_0 for the leftmost sub-region subBlock_0 using a method similar to the conventional method. The region prediction unit 1722c predicts the sub-region subBlock_0 using the reference pixels subRef_0 and the sub-prediction mode subPredMode derived for the sub-region subBlock_0.

[0081] The derivation unit 1722b derives the rightmost column predSample[w-1][y] of predicted pixels of the predicted sub-region subBlock_0 as reference pixels for predicting the sub-region subBlock_1. The region prediction unit 1722c predicts the sub-region subBlock_1 using the derived reference pixels predSample[w-1][y] and the sub-prediction mode subPredMode derived for the sub-region subBlock_1.

[0082] As shown in FIG. 7(b), the derivation unit 1722b calculates the predicted sub-area subBlock The region prediction unit 1722c derives the rightmost column of predicted pixels predSample[2w-1][y] of subregion subBlock_1 as reference pixels for predicting subregion subBlock_2. The region prediction unit 1722c predicts subregion subBlock_2 using the derived reference pixels predSample[2w-1][y] and the sub-prediction mode subPredMode derived for subregion subBlock_2. This operation is repeated up to subregion subBlock_3.

[0083] Next, method 2 will be described. Method 2 has a shorter processing time but lower prediction accuracy than method 1. Specifically, method 2 predicts each of the multiple sub-regions subBlock_i in parallel. The derivation unit 1722b derives reference pixels for each sub-region subBlock_i from decoded reference pixels adjacent to the current block to be coded. The region prediction unit 1722c predicts each of the multiple sub-regions subBlock_i in parallel using the derived reference pixels.

[0084] Specifically, the derivation unit 1722b extends the reference pixels according to the sub-prediction mode subPredMode and the shape of the sub-region subBlock_i. This method can perform parallel calculations without relying on predicted pixel values ​​like Method 1, and therefore can shorten the processing time.

[0085] FIG. 8 is a diagram showing a second method according to this embodiment.

[0086] As shown in FIG. 8, the derivation unit 1722b calculates the value ref_v that approximates the pixel adjacent to the left side of the sub-region subBlock_i as follows.

[0087] The offset value shift_i[y] is calculated as follows.

number

[0088] When i=1,

number

[0089] When i=2,

number

[0090] When i=3,

number

[0091]

number

[0092] The region prediction unit 1722c predicts each of the multiple sub-regions subBlock_i using the sub-prediction mode subPredMode derived by the derivation unit 1722b, and generates multiple prediction regions (predicted regions). Specifically, the region prediction unit 1722c references the reference pixels (subRef_i[x]) derived by the derivation unit 1722b and generates a prediction region predSample[x][y] of the sub-region subBlock_i.

[0093] The region synthesis unit 1722d synthesizes the multiple prediction regions generated by the region prediction unit 1722c and outputs a predicted block (intra-predicted block) for the block to be coded. The region synthesis unit 1722d may perform filtering to reduce discontinuity at the boundaries of the sub-region subBlock_i, or may perform region synthesis by superimposing the prediction regions of the sub-region subBlock_i and performing calculation processing such as taking a weighted average. In this case, the filtering method or calculation processing method may be determined based on the parameters of the sub-region subBlock_i.

[0094] Up to this point, examples have been described in which the block shape of the block to be coded is square and predMode=34...50.

[0095] On the other hand, when the intra prediction mode mode number (predMode) is not 34...50, for example, when predMode>50, the leftmost sub-region must refer to a direction that cannot be predicted within the range of mode numbers 2...66, as shown in Figure 9(a). However, as shown in Figure 9(b), by inverting the current block to be coded, prediction can be performed within the range of mode numbers 2...66. The predicted image generated by inverting in this way can be inverted again to obtain the desired predicted image.

[0096] Furthermore, as shown in FIGS. 10(a) and 10(b), when predMode<18, prediction can be performed in the range of mode numbers 2...66 by rotating by 90 degrees.

[0097] Similarly, when the block shape of the block to be coded is rectangular and the mode number is "-2" to "-14" and the mode number is "67" to "80", a predicted image of the sub-region can be generated within the range of predMode=2...66 by inverting the block to be coded, as shown in Figures 11(a) and 11(b).

[0098] As described above, prediction of a sub-region may be performed after rotating or flipping the coding block according to the mode number of predMode.

[0099] As described above, the encoding device 1 according to this embodiment includes an intra prediction unit 172 (intra prediction device) that performs intra prediction on a block-by-block basis. The intra prediction unit 172 includes a region division unit 1722a that divides a current block, to which an intra prediction mode having a linear prediction direction is assigned, into multiple sub-regions; a derivation unit 1722b that derives a sub-prediction mode for each of the multiple sub-regions according to a reference curve obtained from the linear prediction direction based on the intra prediction mode; a region prediction unit 1722c that predicts each of the multiple sub-regions using the sub-prediction mode to generate multiple prediction regions; and a region synthesis unit 1722d that synthesizes the multiple prediction regions and outputs a predicted block for the current block. This enables highly accurate prediction of curved object boundaries, which is expected to improve encoding efficiency.

[0100] <Configuration of the Decryption Device> Next, the configuration of the decoding device according to this embodiment will be described, focusing on the differences from the encoding device 1. Fig. 12 is a diagram showing the configuration of a decoding device 2 according to this embodiment.

[0101] As shown in FIG. 12, the decoding device 2 includes an entropy code decoding unit 200 , an inverse quantization and inverse transform unit 210 , a synthesis unit 220 , a memory 230 , and a prediction unit 240 .

[0102] The entropy code decoding unit 200 decodes the coded stream (bit stream) and outputs the quantized transform coefficients corresponding to the block to be decoded to the inverse quantization and inverse transform unit 210. The entropy code decoding unit 200 also acquires flag information related to the prediction process from the coded stream and outputs the flag information related to the prediction process to the prediction unit 240.

[0103] Here, the entropy code decoding unit 200 may obtain, from the coded stream, flag information indicating whether or not intra prediction has been applied for each block to be decoded.The entropy code decoding unit 200 may obtain, from the coded stream, flag information indicating whether or not intra curve prediction has been applied for a block to be decoded to which intra prediction has been applied.

[0104] The entropy coding decoding unit 200 may obtain flag information indicating the intra prediction mode predMode of the block to be decoded from the encoding stream, without obtaining flag information indicating the sub prediction mode subPredMode for each sub-region subBlock from the encoding stream, for the block to be decoded to which intra prediction has been applied.

[0105] The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.

[0106] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 inverse quantizes the quantized transform coefficients output from the entropy code decoding unit 200 using a quantization parameter (Qp) and a quantization matrix to restore the transform coefficients of the block to be decoded, and outputs the restored transform coefficients to the inverse transform unit 212.

[0107] The inverse transform unit 212 performs inverse transform processing corresponding to the transform processing performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs inverse transform processing on the transform coefficients output from the inverse quantization unit 211 to restore prediction residuals, and outputs the restored prediction residuals (restored prediction residuals) to the synthesis unit 220.

[0108] The synthesis unit 220 decodes (reconstructs) the original block by synthesizing the prediction residual output from the inverse transform unit 212 and the prediction block output from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded block to the memory 230.

[0109] The memory 230 stores the decoded blocks output from the synthesis unit 220 and accumulates the decoded blocks as decoded images on a frame-by-frame basis. The memory 230 outputs the decoded blocks or the decoded images to the prediction unit 240. The memory 230 also outputs the decoded images on a frame-by-frame basis to the outside of the decoding device 2. Note that a loop filter may be provided between the synthesis unit 220 and the memory 230.

[0110] The prediction unit 240 performs prediction on a block-by-block basis and includes an inter prediction unit 241, an intra prediction unit 242, and a switching unit 243.

[0111] The inter prediction unit 241 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 241 predicts a current block to be decoded using a decoded image stored in the memory 230 as a reference image based on information related to inter prediction (e.g., motion vector information) output from the entropy encoding / decoding unit 200, thereby generating an inter prediction block. The inter prediction unit 241 outputs the generated inter prediction block to the switching unit 243.

[0112] The intra prediction unit 242 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 242 uses an intra prediction mode corresponding to flag information related to intra prediction output from the entropy encoding / decoding unit 200 to generate an intra prediction block by referring to decoded pixels (reference pixels) surrounding a block to be decoded in a decoded image stored in the memory 230, and outputs the generated intra prediction block to the switching unit 243. In this embodiment, the intra prediction unit 242 corresponds to an intra prediction device on the decoding side.

[0113] The switching unit 243 switches between the inter-predicted block output by the inter-prediction unit 241 and the intra-predicted block output by the intra-prediction unit 242, and outputs either the predicted block to the synthesis unit 220.

[0114] Next, the intra prediction unit 242 according to this embodiment will be described.

[0115] The intra prediction unit 242 has a prediction mode specifying unit 2421 that specifies the intra prediction mode predMode to be applied to the block to be decoded, based on flag information acquired by the entropy code decoding unit 200. The intra prediction unit 242 predicts the block to be decoded using the intra prediction mode predMode specified by the prediction mode specifying unit 2421. In this embodiment, the intra prediction mode predMode for each block to be decoded is acquired by the entropy code decoding unit 200, but the intra prediction mode predMode may also be specified on the decoding side based on parameters of the block to be decoded (position, shape, and prediction mode of the neighboring region).

[0116] The intra prediction unit 242 according to this embodiment has an intra curve prediction unit 2422. Fig. 13 is a diagram showing the configuration of the intra curve prediction unit 2422 according to this embodiment. As shown in Fig. 13, the intra curve prediction unit 2422 has a region division unit 2422a, a derivation unit 2422b, a region prediction unit 2422c, and a region synthesis unit 2422d.

[0117] The operations of region dividing section 2422a, derivation section 2422b, region prediction section 2422c, and region combining section 2422d are similar to the operations of region dividing section 1722a, derivation section 1722b, region prediction section 1722c, and region combining section 1722d described above.

[0118] In this way, the decoding device 2 according to this embodiment performs intra-decoding in units of blocks that make up an image. The intra prediction unit 242 includes an intra prediction unit 2422 (intra prediction device) that performs prediction. The intra prediction unit 242 includes a region division unit 2422a that divides a current block, to which an intra prediction mode having a linear prediction direction is assigned, into multiple sub-regions, a derivation unit 2422b that derives a sub-prediction mode for each of the multiple sub-regions according to a reference curve obtained from the linear prediction direction based on the intra prediction mode, a region prediction unit 2422c that predicts each of the multiple sub-regions using the sub-prediction mode to generate multiple prediction regions, and a region synthesis unit 2422d that synthesizes the multiple prediction regions and outputs a predicted block for the current block. This makes it possible to predict curved object boundaries with high accuracy, and is expected to improve coding efficiency.

[0119] <Operation> Next, the operation according to this embodiment will be described, focusing on the operation of the decoding device 2 related to intra curve prediction.

[0120] 14 is a diagram showing the operation of the decoding device 2 according to this embodiment. The decoding device 2 performs this operation for each block to be decoded.

[0121] 14, in step S11, the entropy code decoding unit 200 parses a prediction mode to be applied to a current block based on the coded stream. Specifically, the entropy code decoding unit 200 acquires flag information related to prediction processing from flag information included in the coded stream, and outputs the acquired flag information to the prediction unit 240. This flag information includes an intra prediction flag indicating whether or not intra prediction has been applied, and a curve prediction flag indicating whether or not intra curve prediction has been applied to the current block to which intra prediction has been applied.

[0122] In step S12, the prediction unit 240 determines whether or not the intra prediction flag is "1." If the intra prediction flag is not "1" (step S12: NO), in step S16, the inter prediction unit 241 predicts the current block by inter prediction. The prediction block is output.

[0123] If the intra prediction flag is "1" (step S12: YES), in step S13, the intra prediction unit 242 determines whether the curve prediction flag is "1." In this case, the intra prediction unit 242 acquires flag information indicating the intra prediction mode predMode (for example, flag information indicating the intra prediction mode number) from the entropy encoding / decoding unit 200. This intra prediction mode predMode is assumed to be directional prediction.

[0124] If the curve prediction flag is not "1" (step S13: NO), in step S15, the intra prediction unit 242 predicts the block to be decoded using intra prediction other than intra curve prediction (for example, intra linear prediction) based on the intra prediction mode predMode, and outputs the predicted block.

[0125] If the curve prediction flag is "1" (step S13: YES), in step S14, the intra curve prediction unit 2422 of the intra prediction unit 242 predicts the block to be decoded by intra curve prediction based on the intra prediction mode predMode, and outputs a predicted block.

[0126] FIG. 15 is a diagram showing the operation of the intra curve prediction unit 2422 according to this embodiment.

[0127] 15, in step S21, the region division unit 2422a divides a current block to be decoded, to which an intra-prediction mode predMode having a linear prediction direction is assigned, into a plurality of sub-regions subBlock_i. Here, the region division unit 2422a may determine the division direction and the number of divisions based on parameters (position, shape, and prediction mode) of the current block to be decoded or parameters of adjacent decoded regions. Furthermore, the region division unit 2422a may acquire information on the division shape and position determined on the encoding side from the entropy code decoding unit 200.

[0128] In step S22, as shown in FIG. 6(a), the derivation unit 2422b sets point C0=(0,0) and point C2=(w-1,h-1), and determines the intersection of a line that passes through point C0 and is parallel to the intra prediction mode predMode of the block to be decoded with the boundary line of the block to be decoded as C1=(c x ,c y ) Then, the derivation unit 2422b derives a quadratic Bezier curve with these three points as control points as the reference curve.

[0129] In step S23, the derivation unit 2422b sets, as reference points, the intersections between the boundary lines of each of the sub-regions subBlock_i and the reference curve, as shown in FIG. 6(b).

[0130] In step S24, the derivation unit 2422b derives a sub-prediction mode subPredMode for each of the multiple sub-regions subBlock_i based on each straight line obtained by connecting the reference points. Specifically, as shown in FIG. 6(c), the derivation unit 2422b derives the sub-prediction mode subPredMode for each sub-region subBlock_i by selecting the intra prediction mode (see FIG. 3) having the slope closest to the slope of the straight line obtained by connecting the reference points. Note that, although the slope of the straight line is the slope between two points here, the slope may also be obtained using a method such as differentiation of the curve at the reference point.

[0131] In step S25, the derivation unit 2422b derives reference pixels for predicting the sub-region subBlock using the derived sub-prediction mode subPredMode. The reference pixels are extended using the above-mentioned method 1 or method 2. The region prediction unit 2422c then predicts each of the multiple sub-regions subBlock_i using the sub-prediction mode subPredMode derived by the derivation unit 2422b and the reference pixels, and generates multiple prediction regions (predicted regions).

[0132] In step S26, the region synthesis unit 2422d synthesizes the multiple prediction regions generated by the region prediction unit 2422c and outputs a predicted block (intra-predicted block) for the block to be decoded. The region synthesis unit 2422d may perform filtering to reduce discontinuity at the boundaries of the sub-region subBlock_i, or may perform region synthesis by superimposing the prediction regions of the sub-region subBlock_i and calculating a weighted average, for example. In this case, the filtering method or calculation method may be determined based on the parameters of the sub-region subBlock_i.

[0133] 16 and 17 are diagrams showing examples of prediction by the intra curve prediction unit 2422 according to this embodiment. Here, the target block is divided into four vertically. In FIG. 16, the intra prediction mode predMode is mode "24," and in FIG. 17, the intra prediction mode predMode is mode "42." Also, it is assumed that the target block size is 32×32, the size of the sub-region subBlock_i is 8×32 pixels (divided into four vertically), and the reference curve is a quadratic Bezier curve.

[0134] 16(a) and 17(a) show examples of prediction when intra prediction (i.e., intra linear prediction) is performed on a current block using an intra prediction mode predMode having a linear prediction direction. As shown in FIG. 16(a) and FIG. 17(a), intra linear prediction performs prediction by linearly copying some of the reference pixels.

[0135] 16(b) and 17(b) are diagrams illustrating a comparison between the reference curve and the sub-prediction mode in the intra curve prediction according to this embodiment and a linear prediction direction. As shown in Fig. 16(b) and 17(b), the intra curve prediction unit 2422 derives a sub-prediction mode subPredMode[i] corresponding to the reference curve obtained from the linear prediction direction for each sub-region subBlock_i, predicts each sub-region subBlock_i using the sub-prediction mode subPredMode[i] to generate multiple prediction regions, and combines the multiple prediction regions to output a prediction block for the current block to be coded.

[0136] 16(c) and 17(c) are diagrams showing predicted blocks by intra curve prediction according to this embodiment. As shown in FIG. 16(c) and FIG. 17(c), curved prediction is performed in predicted blocks by intra curve prediction. This makes it possible to predict curved object boundaries with high accuracy, and is expected to improve coding efficiency.

[0137] <Other embodiments> In the above embodiment, an example has been described in which a block to be coded is divided into regions and prediction is performed for each region, but prediction may also be performed line by line, in which case each line can be regarded as a sub-region to be predicted.

[0138] In the above embodiment, an example has been described in which the reference curve is a Bezier curve, but a different curve may be used based on the position, shape, and prediction mode of the block to be coded.

[0139] In the above embodiment, an example has been described in which prediction residuals for the entire block to be coded obtained by combining sub-regions are calculated and transformed, but prediction residuals may be calculated and transformed for each sub-region. Specifically, prediction processing may be performed according to an intra prediction mode determined for each sub-region, and transformation processing may be performed on the prediction residuals for each sub-region. Furthermore, by performing quantization, inverse quantization, and inverse transformation for each sub-region, the prediction residuals for that sub-region are restored, and by combining them with the prediction residuals for the sub-region, it becomes possible to sequentially obtain decoded images for that sub-region. In prediction processing for subsequent sub-regions, the decoded image for that sub-region is used to restore the prediction residuals for that sub-region. May be used in prediction processing

[0140] In the above-described embodiment, an example has been described in which a sub-prediction mode for each sub-region is derived based on a straight line obtained by connecting each boundary line of the sub-region with each intersection point of the reference curve. However, a reference position may be set for each sub-region, and the sub-prediction mode for each sub-region may be derived according to the slope of the reference curve at this reference position. Figure 18 shows sub-prediction modes when the reference position is the intersection point of the left side of each sub-region with the reference curve.

[0141] In the above example, the reference position is the intersection point between the left side of each sub-region and the reference curve, but this is not limited to this. For example, the reference position may be the point at which the reference curve is located at the center of the sub-region in the x-axis direction, or the intersection point between the right side of the sub-region and the reference curve. By setting the same number of reference positions on the reference curve as the number of divisions, it is possible to obtain the required number of sub-prediction modes. The slope of the curve at the reference position can be derived by calculating the derivative of the reference curve at the reference position.

[0142] In the above embodiment, when encoding a block adjacent to the encoding target block, The prediction mode of the block may be used as a candidate prediction mode. For example, the prediction mode of the sub-region closest to the adjacent block may be used as a candidate prediction mode, or the average value of multiple prediction modes may be used, or the prediction mode of the block may be used as a candidate prediction mode.

[0143] A program may be provided that causes a computer to execute each process performed by the encoding device 1. A program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium makes it possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0144] The circuits that execute the processes performed by the encoding device 1 are integrated, and the encoding device 1 is implemented as a semiconductor integrated circuit. The decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC) by integrating circuits that execute the processes performed by the decoding device 2.

[0145] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention. [Explanation of symbols]

[0146] 1: Encoding device 2: Decryption device 100: Block division section 110: Subtraction section 120: Transformation and quantization unit 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization section 142: Inverse conversion unit 150: Synthesis section 160: Memory 170: Prediction Department 171: Inter prediction unit 172: Intra prediction unit 173: Switching section 200: Entropy coding and decoding unit 210: Inverse quantization and inverse transformation unit 211: Inverse quantization section 212: Inverse conversion unit 220: Synthesis section 230: Memory 240: Prediction section 241: Inter prediction unit 242: Intra prediction unit 243: Switching section 1721: Prediction mode determination unit 1722: Intra curve prediction unit 1722a: Area division part 1722b: Derivation part 1722c: Area prediction section 1722d: Area synthesis section 2421: Prediction mode identification unit 2422: Intra curve prediction unit 2422a: Area division part 2422b: Derivation part 2422c: Area prediction section 2422d: Area synthesis section

Claims

1. A prediction device that performs prediction on a block-by-block basis that constitutes an image, a region dividing unit that divides a current block, to which a prediction mode that controls intra prediction processing is assigned, into a plurality of sub-regions; a derivation unit that derives sub-prediction modes for the plurality of sub-regions in accordance with the prediction mode, a block shape of the current block, and a division shape of the sub-regions; a region prediction unit that predicts each of the plurality of sub-regions using the sub-prediction mode to generate a plurality of prediction regions; an area synthesis unit that synthesizes the plurality of prediction areas and outputs a predicted block for the current block.

2. When sequentially predicting each of the plurality of sub-regions, the derivation unit derives predicted pixels included in the predicted sub-region as reference pixels; The intra prediction device according to claim 1 , wherein the region prediction unit predicts an unpredicted sub-region adjacent to the predicted sub-region using the derived reference pixels.

3. When predicting each of the plurality of sub-regions in parallel, the derivation unit derives reference pixels for each of the sub-regions from decoded reference pixels in the periphery of the current block; The intra prediction device according to claim 1 , wherein the region prediction unit predicts each of the plurality of sub-regions in parallel using the derived reference pixels.

4. An encoding device comprising the intra prediction device according to any one of claims 1 to 3.

5. further comprising an entropy coding unit that outputs a coded stream; The encoding device according to claim 4 , wherein the entropy encoding unit does not include flag information indicating the sub-prediction mode in the encoded stream, but includes flag information indicating the prediction mode in the encoded stream.

6. A decoding device comprising the intra prediction device according to any one of claims 1 to 3.

7. further comprising an entropy code decoding unit that decodes the coded stream; The decoding device according to claim 6, characterized in that the entropy code decoding unit does not obtain flag information indicating the sub-prediction mode from the encoded stream, but obtains flag information indicating the prediction mode from the encoded stream.

8. A program that causes a computer to function as the intra prediction device according to any one of claims 1 to 3.

Images