Intra predictor, coding device, decoding device, and program

Abstract

To improve encoding efficiency in a case of performing directivity prediction.SOLUTION: An intra prediction device for performing intra prediction in a block unit constituting an image comprises: an area division unit 1722a which divides an object block to which an intra prediction mode having a linear prediction direction is allocated into a plurality of sub areas; a derivation unit 1722b which derives a sub prediction mode for each of the plurality of sub areas on the basis of a reference curve obtained according to the intra prediction mode, the block shape of the object block and the division number of the object block; and an area prediction unit 1722c which generates a plurality of prediction areas by predicting each of the plurality of sub areas using the sub prediction mode; and an area composition unit 1722d which outputs a prediction block with respect to the object block by combining the plurality of prediction areas.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an intra prediction device, an encoding device, a decoding device, and a program.

Background Art

[0002] In order to transmit a video signal with a huge amount of information in a limited bandwidth, it is essential to perform encoding. In the conventional video encoding method and the next-generation video encoding method (VVC) that are standardized, a frame is divided into blocks, which are small regions (hereinafter referred to as encoding target blocks), and prediction and encoding are performed for each block.

[0003] Prediction methods are roughly classified into two methods: intra prediction and inter prediction. Intra prediction is a method of performing prediction using spatial correlation by using decoded blocks adjacent to the encoding target block within a frame. Inter prediction is a method of performing prediction using temporal correlation by using a decoded frame different from the frame to which the encoding target block belongs. Generally, in a moving image, frames that are temporally close to each other are similar. Therefore, compared with intra prediction, the prediction accuracy of inter prediction is higher, and the amount of information of the residual signal (prediction residual) transmitted to the decoding side is smaller.

[0004] On the other hand, in intra prediction, it is difficult to perform highly accurate prediction compared with inter prediction. Therefore, the amount of information of the residual signal transmitted to the decoding side is overwhelmingly larger than that of inter prediction. In order to transmit the residual signal to which intra prediction is applied in a limited bandwidth, coarser quantization is required, so that the deterioration of the blocks encoded and decoded by intra prediction is likely to occur. Since the image decoded by intra prediction is referred to during inter prediction, the deterioration caused by intra prediction also affects the inter prediction frame that refers to this intra prediction frame, causing deterioration of the entire video. For the above reasons, an encoding method that can compress intra prediction frames with high efficiency has been studied.

[0005] VVC's intra-prediction introduces Planar prediction, which performs surface prediction using decoded reference pixels adjacent to the target block (hereinafter simply referred to as "reference pixels"), DC prediction, which performs prediction using the average value of the reference pixels, and directional prediction, which performs prediction by linearly copying a portion of the reference pixels. In VVC's directional prediction, the number of prediction directions has increased to 65, and 65 prediction modes are defined. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] JVET-S2001 Versatile Video Coding (Draft 10) [Disclosure of the Invention] [Problems that the invention aims to solve]

[0007] While VVC's directional prediction improved accuracy by increasing the number of prediction directions to 65, all directional prediction modes still perform predictions linearly. However, in general natural images, object boundaries are often curved, and linear prediction cannot accurately predict these object boundaries, leading to a decrease in coding efficiency.

[0008] To accurately predict curvilinear object boundaries, it is necessary to subdivide the blocks to be encoded at the object boundary and apply the most accurate directional prediction to each of these sub-blocks. However, as the block subdivision becomes finer, the amount of flag information related to the block subdivision increases, which may reduce encoding efficiency.

[0009] Therefore, the present invention aims to provide an intra-prediction device, an encoding device, a decoding device, and a program that can improve coding efficiency when performing direction prediction. [Means for solving the problem]

[0010] The intra-prediction device according to the first embodiment is an intra-prediction device that performs intra-prediction on a block-by-block basis that constitutes an image, and comprises: a region division unit that divides a target block to which an intra-prediction mode having a linear prediction direction is assigned into a plurality of sub-regions; a derivation unit that derives a sub-prediction mode for each of the plurality of sub-regions based on a reference curve determined according to the intra-prediction mode, the block shape of the target block, and the number of divisions of the target block; a region prediction unit that generates a plurality of prediction regions by predicting each of the plurality of sub-regions using the sub-prediction modes; and a region synthesis unit that synthesizes the plurality of prediction regions and outputs a prediction block for the target block.

[0011] The coding device according to the second embodiment is characterized in that it includes an intra-prediction device according to the first embodiment.

[0012] The decoding device according to the third embodiment is characterized in that it includes an intra-prediction device according to the first embodiment.

[0013] The program according to the fourth embodiment essentially enables a computer to function as an intra-prediction device according to the first embodiment. [Effects of the Invention]

[0014] According to the present invention, it is possible to provide an intra-prediction device, an encoding device, a decoding device, and a program that can improve the coding efficiency when performing direction prediction. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows the configuration of the encoding device according to the embodiment. [Figure 2] This is a diagram illustrating the operation of the intra prediction unit according to the embodiment. [Figure 3] This figure shows the intra prediction mode of the intra prediction unit according to the embodiment. [Figure 4] It is a diagram showing the configuration of the intra curve prediction unit of the encoding device according to the embodiment. [Figure 5] It is a diagram showing the operation of the region division unit according to the embodiment. [Figure 6] It is a diagram showing the operation of deriving the sub-prediction mode according to the embodiment. [Figure 7] It is a diagram showing Method 1 according to the embodiment. [Figure 8] It is a diagram showing Method 2 according to the embodiment. [Figure 9] It is a diagram showing another method according to the embodiment. [Figure 10] It is a diagram showing another method according to the embodiment. [Figure 11] It is a diagram showing another method according to the embodiment. [Figure 12] It is a diagram showing the configuration of the decoding device according to the embodiment. [Figure 13] It is a diagram showing the configuration of the intra curve prediction unit of the decoding device according to the embodiment. [Figure 14] It is a diagram showing the operation of the decoding device according to the embodiment. [Figure 15] It is a diagram showing the operation of the intra curve prediction unit according to the embodiment. [Figure 16] It is a diagram showing a prediction example by the intra curve prediction unit according to the embodiment. [Figure 17] It is a diagram showing a prediction example by the intra curve prediction unit according to the embodiment. [Figure 18] It is a diagram showing another embodiment.

Embodiments for Carrying Out the Invention

[0016] Referring to the drawings, an encoding device and a decoding device according to an embodiment will be described. The encoding device and the decoding device according to the embodiment respectively perform encoding and decoding of moving images represented by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] <Configuration of the encoding device> First, the configuration of the encoding device according to this embodiment will be described. Figure 1 is a diagram showing the configuration of the encoding device 1 according to this embodiment.

[0018] As shown in Figure 1, the encoding device 1 includes a block division unit 100, a subtraction unit 110, a transformation / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transformation unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.

[0019] The block division unit 100 divides the original image, which is an input image in units of frames (or pictures) that constitute the moving image, into multiple image blocks, and outputs the image blocks obtained by the division to the subtraction unit 110. The size of the image block is, for example, 32 x 32 pixels, 16 x 16 pixels, 8 x 8 pixels, or 4 x 4 pixels. The shape of the image block is not limited to a square; it may also be rectangular (non-square).

[0020] An image block is a unit that the encoding device 1 performs encoding on (i.e., the block to be encoded) and a unit that the decoding device performs decoding on (i.e., the block to be decoded). Such an image block is sometimes called a CU (Coding Unit). When there is no particular distinction between the block to be encoded and the block to be decoded, they are simply called target blocks.

[0021] The block division unit 100 outputs luminance blocks and chrominance blocks by performing block division on the luminance signal and chrominance signal that constitute the image. The division of the luminance signal and chrominance signal may be controlled independently. When luminance blocks and chrominance blocks are not specifically distinguished, they are simply called the encoded blocks.

[0022] The subtraction unit 110 calculates a predicted residual, which represents the difference (error) between the block to be encoded output from the block division unit 100 and the predicted block obtained by the prediction unit 170 predicting the block to be encoded. Specifically, the subtraction unit 110 calculates the predicted residual by subtracting each pixel value of the predicted block from each pixel value of the block, and outputs the calculated predicted residual to the conversion / quantization unit 120.

[0023] The conversion and quantization unit 120 performs conversion and quantization processing in block units. The conversion and quantization unit 120 comprises a conversion unit 121 and a quantization unit 122.

[0024] The conversion unit 121 performs a conversion process on the predicted residual output from the subtraction unit 110 to calculate conversion coefficients, and outputs the calculated conversion coefficients to the quantization unit 122. Conversion processing refers to, for example, discrete cosine transform (DCT), discrete sine transform (DST), Carunen-Löwe ​​transform (KLT), etc. Conversion processing may include conversion skipping, which adjusts the signal of a pixel block by scaling, etc., without converting it to a frequency domain signal.

[0025] The quantization unit 122 quantizes the conversion coefficients output from the conversion unit 121 using the quantization parameter (Qp) and the quantization matrix, and outputs the quantized conversion coefficients to the entropy coding unit 130 and the inverse quantization / inverse conversion unit 140.

[0026] The entropy coding unit 130 performs entropy coding on the conversion coefficients output from the quantization unit 122, compresses the data, generates a coded stream (bitstream), and outputs the coded stream to the outside of the coding device 1. For entropy coding, Huffman coding or CABAC (Context-based Adaptive Binary Arithmetic Coding) can be used.

[0027] The inverse quantization / inverse transformation unit 140 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization / inverse transformation unit 140 comprises an inverse quantization unit 141 and an inverse transformation unit 142.

[0028] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the transformation coefficients output from the quantization unit 122 by inverse quantization using the quantization parameter (Qp) and the quantization matrix, and outputs the restored transformation coefficients to the inverse transformation unit 142.

[0029] The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform process on the transform coefficients output from the inverse quantization unit 141 to restore the predicted residuals, and outputs the restored predicted residuals to the synthesis unit 150.

[0030] The synthesis unit 150 synthesizes the reconstructed predicted residuals output from the inverse transform unit 142 with the predicted blocks output from the prediction unit 170 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the reconstructed predicted residuals to each pixel value of the predicted blocks to decode (reconstruct) the blocks to be encoded, and outputs the decoded blocks to the memory 160. The decoded blocks are sometimes called reconstructed blocks.

[0031] Memory 160 stores the decoded blocks output from the synthesis unit 150 and accumulates the decoded blocks as decoded images on a frame-by-frame basis. Memory 160 outputs the stored decoded blocks or decoded images to the prediction unit 170. A loop filter may be provided between the synthesis unit 150 and memory 160.

[0032] The prediction unit 170 performs prediction processing in block units. The prediction unit 170 includes an inter-prediction unit 171, an intra-prediction unit 172, and a switching unit 173. In addition to intra-prediction (intra-linear prediction) and inter-prediction similar to conventional methods, the prediction unit 170 calculates the RD cost of intra-curve prediction, determines the optimal prediction mode, and generates prediction blocks in that optimal prediction mode.

[0033] The interpretation unit 171 performs interpretation using correlations between frames. Specifically, the interpretation unit 171 uses the decoded image stored in memory 160 as a reference image, calculates motion vectors using methods such as block matching, predicts the blocks to be encoded, generates interpretation blocks, and outputs the generated interpretation blocks to the switching unit 173. Here, the interpretation unit 171 selects the optimal interpretation method from among interpretation using multiple reference images (typically biprediction) and interpretation using one reference image (unidirectional prediction), and performs interpretation using the selected interpretation method. The interpretation unit 171 outputs information related to interpretation (motion vectors, etc.) to the entropy coding unit 130.

[0034] The intra-prediction unit 172 performs intra-prediction using spatial correlations within the frame. Specifically, the intra-prediction unit 172 generates intra-prediction blocks by referring to reference pixels, which are decoded pixels surrounding the encoding target block, from the decoded image stored in the memory 160, and outputs the generated intra-prediction blocks to the switching unit 173. In this embodiment, the intra-prediction unit 172 corresponds to the intra-prediction device on the encoding side.

[0035] The intra-prediction unit 172 includes a prediction mode determination unit 1721 that determines the intra-prediction mode predMode to be applied to the block to be encoded from among a plurality of intra-prediction modes. The intra-prediction unit 172 predicts the block to be encoded using the intra-prediction mode predMode determined by the prediction mode determination unit 1721. The intra-prediction unit 172 outputs flag information indicating the determined intra-prediction mode predMode to the entropy coding unit 130. In this embodiment, the intra-prediction mode predMode for each block to be encoded is signaled to the decoding side by the entropy coding unit 130, but the decoding side may determine the intra-prediction mode predMode based on the parameters of the block to be encoded (position, block shape, and prediction mode of adjacent regions).

[0036] The intra-prediction unit 172 in this embodiment includes an intra-curve prediction unit 1722. Details of the intra-curve prediction unit 1722 will be described later.

[0037] The switching unit 173 switches between the inter-prediction block output from the inter-prediction unit 171 and the intra-prediction block output from the intra-prediction unit 172, and outputs one of the prediction blocks to the subtraction unit 110 and the synthesis unit 150.

[0038] Next, the intra-prediction unit 172 according to this embodiment will be described. Figure 2 is a diagram illustrating the operation of the intra-prediction unit 172 according to this embodiment.

[0039] As shown in Figure 2, the intra-prediction unit 172 performs intra-prediction using one of the following intra-prediction modes: Planar prediction, which performs surface prediction using decoded reference pixels adjacent to the block to be encoded; DC prediction, which performs prediction using the average value of the reference pixels; and directional prediction, which performs prediction by linearly copying a portion of the reference pixels. The reference pixels are selected from the upper and leftmost reference pixel rows of the block to be encoded according to the intra-prediction mode. In this embodiment, there are 65 prediction directions in directional prediction, i.e., 65 different intra-prediction modes are available.

[0040] In the following explanation, we will use the example of a case where the available reference pixels are decoded pixel signals adjacent to the upper or left side of the block to be encoded. However, if multi-reference line prediction, which determines the position of the reference pixel to be used for prediction processing from among the multiple reference pixel sequences introduced in the VVC standard, is introduced, prediction processing may be performed using a reference pixel that is not adjacent to the block to be encoded, for example, a reference pixel located one pixel away from the block to be encoded.

[0041] Figure 3 shows the intra prediction mode of the intra prediction unit 172 according to this embodiment.

[0042] As shown in Figure 3, there are 67 candidate intra-prediction modes, ranging from 0 to 66. Intra-prediction mode "0" is planar prediction, intra-prediction mode "1" is DC prediction, and intra-prediction modes "2" through "66" are directional prediction. In directional prediction, the direction of the arrow indicates the reference direction, the starting point of the arrow indicates the position of the pixel to be predicted, and the ending point of the arrow indicates the position of the reference pixel used to predict this pixel.

[0043] There are two intra-prediction modes: Mode "2," which refers to the lower left direction as a reference direction parallel to the diagonal line passing through the upper right and lower left vertices of the block to be encoded, and Mode "66," which refers to the upper right direction. Mode numbers are assigned clockwise at predetermined angle intervals from Mode "2" to Mode "66." Mode numbers "-2" to "-14" and Mode numbers "67" to "80" are applicable only when the block shape of the block to be encoded is rectangular (not square). Below, we will mainly explain an example where the block shape of the block to be encoded is square and the mode numbers are "34" to "50."

[0044] Next, an intra-curve prediction unit 1722 according to one embodiment will be described. As mentioned above, general directionality prediction is performed by linearly copying a part of the reference pixels. However, in general natural images, object boundaries are often curved, and linear prediction cannot accurately predict these object boundaries, resulting in a problem of reduced coding efficiency.

[0045] The intra-curve prediction unit 1722 according to this embodiment divides the target block to be encoded, to which an intra-prediction mode predMode having a linear prediction direction is assigned, into multiple sub-regions. It derives a sub-prediction mode for each of the multiple sub-regions corresponding to a reference curve obtained from the linear prediction direction. Using the sub-prediction modes, it predicts each of the multiple sub-regions to generate multiple prediction regions. The multiple prediction regions are then combined to output a prediction block for the target block to be encoded. This makes it possible to predict curvilinear object boundaries with high accuracy, and an improvement in encoding efficiency can be expected.

[0046] In this embodiment, by performing intra-curve prediction using a common algorithm on both the encoding and decoding sides, signaling of flag information indicating sub-prediction modes becomes unnecessary, and the intra-prediction mode predMode for each block to be encoded can be signaled as in the conventional method. Specifically, the entropy encoding unit 130 includes flag information indicating the intra-prediction mode predMode in the encoding stream, but does not include flag information indicating sub-prediction modes in the encoding stream. This suppresses an increase in the amount of flag information to be signaled to the decoding side, and prevents a decrease in encoding efficiency due to the amount of flag information.

[0047] The entropy coding unit 130 may include flag information in the coding stream indicating whether or not intra prediction was applied for each block to be coded. The entropy coding unit 130 may also include flag information in the coding stream indicating whether or not intra curve prediction was applied for blocks to be coded to which intra prediction was applied.

[0048] Furthermore, the prediction process may be controlled using existing flags. For example, the flag information indicating whether intra curve prediction is applicable may be shared with the flag information indicating whether ISP mode is applicable in VVC (intra_subpartitions_mode_flag), and only when ISP mode is applicable, a flag information indicating whether ISP mode was applied or intra curve prediction was applied (e.g., single_prediction_mode_flag, True for conventional ISP mode, False for intra curve prediction) may be added to determine whether curve intra prediction was applied. In this case, the flag information indicating the division direction of curve intra prediction may also be shared with the flag information indicating the division direction of ISP mode (intra_subpartitions_split_flag).

[0049] Figure 4 shows the configuration of the intra-curve prediction unit 1722 according to this embodiment. As shown in Figure 4, the intra-curve prediction unit 1722 includes a region division unit 1722a, a derivation unit 1722b, a region prediction unit 1722c, and a region synthesis unit 1722d.

[0050] Figure 5 shows the operation of the region division unit 1722a according to this embodiment. As shown in Figure 5, the region division unit 1722a divides the encoding target block to which an intra prediction mode predMode having a linear prediction direction is assigned into a plurality of sub-regions subBlock_i (i=0,1,…,n).

[0051] In this embodiment, an example is described in which the region division unit 1722a divides the block to be encoded into four vertical sections. However, the division direction and number of divisions may be determined based on the parameters of the block to be encoded (position, block shape, and prediction mode) or the parameters of adjacent decoded regions. For example, the block to be encoded may be divided into a quadtree in a grid pattern, or it may be divided horizontally or vertically to a predetermined number of blocks. In addition, information on the division shape and position of each sub-region determined on the encoding side may be signaled from the entropy encoding unit 130 to the decoding side. Furthermore, the division shape does not have to be rectangular; it may be divided into triangular or trapezoidal regions as introduced in VVC.

[0052] The derivation unit 1722b derives a subprediction mode subPredMode for each of the multiple sub-regions subBlock_i, based on the intraprediction mode predMode, according to a reference curve obtained from the linear prediction direction of the intraprediction mode predMode.

[0053] Firstly, the derivation unit 1722b derives the subprediction mode subPredMode for each of the multiple sub-regions subBlock_i. In this embodiment, the derivation unit 1722b sets each intersection point between the boundary line of each of the multiple sub-regions subBlock_i and the reference curve as a reference point, and derives the subprediction mode subPredMode for each of the multiple sub-regions subBlock_i based on the straight lines obtained by connecting the reference points.

[0054] Figure 6 shows the derivation process of the subprediction mode subPredMode according to this embodiment.

[0055] As shown in Figure 6(a), the derivation unit 1722b sets point C0 = (0,0) and point C2 = (w-1,h-1). Here, "w" represents the width of the block to be encoded, and "h" represents the height of the block to be encoded. The derivation unit 1722b also sets the intersection point of the line passing through point C0 and parallel to the intra-prediction mode predMode of the block to be encoded and the boundary line of the block to be encoded as C1 = (c x ,cy )

[0056] As shown in Figure 6(b), the derivation unit 1722b uses a quadratic Bézier curve with these three points as control points as the reference curve, and the intersection of the reference curve and the dividing line as the reference point. In this embodiment, the reference curve is a quadratic Bézier curve, but other curves may be used, or the system may maintain a table of corresponding prediction parameters according to the intra prediction mode predMode and the division shape, and refer to the table.

[0057] The reference point is determined from the prediction mode of the block to be encoded and the block shape (for example, the width W and height H of the block to be encoded) as follows. First, the control point C1 is determined using the intraPredAngle defined in VVC as follows.

[0058]

number

[0059] Next, the Bezier curve Bezier[t] can be calculated as follows:

[0060]

number

[0061] The reference point can be obtained by solving Bezier[t] for t, but a table may be stored in advance depending on the prediction mode and block shape of the block to be encoded. The y coordinate of the reference point of the obtained sub-region subBlock_i is b i Let's assume that.

[0062] As shown in Figure 6(c), the derivation unit 1722b derives the subprediction mode subPredMode for each sub-region subBlock_i by selecting the intraprediction mode (see Figure 3) that has the closest slope to the slope of the straight line obtained by connecting the reference points. Here, the slope of the straight line is taken as the slope between two points, but the slope may also be determined using methods such as differentiation of the curve at the reference points.

[0063] For example, the derivation section 1722b calculates the slope[i] of the i-th sub-region subBlock_i.

[0064]

number

[0065] It is calculated using the formula, where w = W / 4.

[0066] Furthermore, a table called slopeTable, which associates the slope with the prediction mode, slopeTable = [0, 1, 2, 3, 4, 6, 6, 8, 8, 10, 10, 12, 12, 14, 14, 16, 16, 18, 18, 20, 20, 20, 23, 23, 23, 26, 26, 26, 29, 29, 29, 32, 32] Therefore, the prediction mode subPredMode[i] of the i-th sub-region subBlock_i is:

[0067]

number

[0068] It can be determined by this method.

[0069] Secondly, the derivation unit 1722b derives reference pixels for predicting the sub-region subBlock using the derived sub-prediction mode subPredMode.

[0070] When the derivation unit 1722b derives the reference pixel subRef_0[x] of the leftmost sub-region subBlock_0, it derives the reference pixel using the same method as the conventional method, as follows.

[0071] subRef_0[ x ] = p[ -1 + x ][ -1 ] (0 ≦ x ≦ iw+1) subRef_0[ x ] = p[ -1][ -1 + Min( ( x * invAngle + 256 ) >> 9, h ) (-H ≦ x ≦ -1 )

[0072] When the derivation unit 1722b derives the reference pixels (subBlock_i[x], i=1,2,3) of sub-regions subBlock_i other than the leftmost column, it derives the reference pixels as follows.

[0073] subRef_i[x] = p[ iw+x-1 ][ -1 ] ( 0 ≦ x ≦ iw+1 )

[0074] Here, the reference pixels at x = -H...-1 are expanded according to the sub-prediction modes subPredMode from subBlock_0 to subBlock_i-1. There are two methods for this expansion.

[0075] First, let's explain Method 1. Method 1 is capable of high-precision prediction, but it is a method with a long processing time. Specifically, Method 1 sequentially predicts each of multiple sub-regions subBlock_i. The derivation unit 1722b derives the predicted pixels included in the predicted sub-regions subBlock_i as reference pixels. The region prediction unit 1722c uses the derived reference pixels to predict the unpredicted sub-regions subBlock_i adjacent to the predicted sub-regions subBlock_i.

[0076] Figure 7 shows Method 1 according to this embodiment.

[0077] As shown in Figure 7, the derivation unit 1722b expands the reference pixels according to the subprediction mode subPredMode and the pixel values ​​of the predicted subregion subBlock_i. This method is expected to provide highly accurate predictions because it expands the reference pixels of adjacent subregions subBlock_i using the pixel values ​​of the predicted subregion subBlock_i. Furthermore, it can be easily implemented using algorithms used in conventional encoding schemes.

[0078] subRef_i[x] is obtained using the rightmost column predSample[iw-1][y] of the predicted pixels of the sub-region subBlock_i-1 predicted by the region prediction unit 1722c, as follows:

[0079] subRef_i[x] = predSample[iw-1][ -1+ Min( ( x * invAngle + 256 ) >> 9, H )] (-H-1 ≤ x ≤ -2)

[0080] As shown in Figure 7(a), the derivation unit 1722b derives the reference pixel subRef_0 for the leftmost sub-region subBlock_0 using the same method as the conventional method. The region prediction unit 1722c predicts the sub-region subBlock_0 using the reference pixel subRef_0 and the sub-prediction mode subPredMode derived for the sub-region subBlock_0.

[0081] The derivation unit 1722b derives the rightmost column of predicted pixels in the predicted sub-region subBlock_0, predSample[w-1][y], as a reference pixel for predicting sub-region subBlock_1. The region prediction unit 1722c uses the derived reference pixel predSample[w-1][y] and the sub-prediction mode subPredMode derived for sub-region subBlock_1 to predict sub-region subBlock_1.

[0082] As shown in Figure 7(b), the derivation unit 1722b derives the rightmost column of predicted pixels in the predicted sub-region subBlock_1, predSample[2w-1][y], as a reference pixel for predicting sub-region subBlock_2. The region prediction unit 1722c uses the derived reference pixel predSample[2w-1][y] and the sub-prediction mode subPredMode derived for sub-region subBlock_2 to predict sub-region subBlock_2. This operation is repeated up to sub-region subBlock_3.

[0083] Next, we will explain Method 2. Although Method 2 has a shorter processing time, its prediction accuracy is lower than that of Method 1. Specifically, Method 2 predicts each of the multiple sub-regions subBlock_i in parallel. The derivation unit 1722b derives reference pixels for each sub-region subBlock_i from the decoded reference pixels adjacent to the block to be encoded. The region prediction unit 1722c uses the derived reference pixels to predict each of the multiple sub-regions subBlock_i in parallel.

[0084] Specifically, the derivation unit 1722b expands the reference pixels according to the subprediction mode subPredMode and the shape of the subregion subBlock_i. This method can be calculated in parallel without depending on the predicted pixel values, as in Method 1, thus reducing processing time.

[0085] Figure 8 shows method 2 according to this embodiment.

[0086] As shown in Figure 8, the derivation unit 1722b calculates the approximate value ref_v, which is the pixel adjacent to the left side of the sub-region subBlock_i, as follows.

[0087] The offset value shift_i[y] is calculated as follows:

number

[0088] When i=1,

number

[0089] When i=2,

number

[0090] When i=3,

number

[0091]

number

[0092] The region prediction unit 1722c predicts each of the multiple sub-regions subBlock_i using the sub-prediction mode subPredMode derived by the derivation unit 1722b, and generates multiple predicted regions (predicted regions). Specifically, the region prediction unit 1722c refers to the reference pixel (subRef_i[x]) derived by the derivation unit 1722b and generates the predicted region predSample[x][y] of the sub-region subBlock_i.

[0093] The region merging unit 1722d merges multiple predicted regions generated by the region prediction unit 1722c and outputs a predicted block (intra-predicted block) for the block to be encoded. The region merging unit 1722d may apply filtering to reduce discontinuities at the boundaries of sub-regions subBlock_i, or it may perform region merging by superimposing the predicted regions of sub-regions subBlock_i and performing computational processing such as taking a weighted average. In this case, the filtering method and computational processing method may be determined based on the parameters of sub-regions subBlock_i.

[0094] Up to this point, we have mainly described examples where the block shape of the block to be encoded is square and predMode=34...50.

[0095] On the other hand, if the mode number (predMode) of the intra-prediction mode is not 34...50, for example, when predMode > 50, as shown in Figure 9(a), the leftmost sub-region must refer to directions that cannot be predicted within the range of mode numbers 2...66. However, as shown in Figure 9(b), by inverting the encoding target block, prediction can be performed within the range of mode numbers 2...66. The predicted image generated by this inversion can then be inverted again to obtain the desired predicted image.

[0096] Furthermore, as shown in Figures 10(a) and 10(b), when predMode < 18, predictions can be made in the range of mode numbers 2...66 by rotating by 90 degrees.

[0097] Similarly, when the block shape of the block to be encoded is rectangular, and the mode numbers are "-2" to "-14" and "67" to "80", as shown in Figures 11(a) and 11(b), a prediction image of a sub-region can be generated within the range of predMode=2...66 by inverting the block to be encoded.

[0098] As described above, the coding block may be rotated and flipped according to the mode number of predMode before predicting the sub-region.

[0099] As described above, the encoding device 1 according to this embodiment has an intra-prediction unit 172 (intra-prediction device) that performs intra-prediction on a block-by-block basis that constitutes an image. The intra-prediction unit 172 includes a region division unit 1722a that divides a target block to which an intra-prediction mode having a linear prediction direction is assigned into a plurality of sub-regions, a derivation unit 1722b that derives a sub-prediction mode for each of the plurality of sub-regions according to a reference curve obtained from the linear prediction direction based on the intra-prediction mode, a region prediction unit 1722c that generates a plurality of prediction regions by predicting each of the plurality of sub-regions using the sub-prediction modes, and a region synthesis unit 1722d that synthesizes the plurality of prediction regions and outputs a prediction block for the target block. This makes it possible to predict curved object boundaries with high accuracy and is expected to improve encoding efficiency.

[0100] <Configuration of the decryption device> Next, the configuration of the decoding device according to this embodiment will be explained, mainly focusing on the differences from the encoding device 1. Figure 12 is a diagram showing the configuration of the decoding device 2 according to this embodiment.

[0101] As shown in Figure 12, the decoding device 2 includes an entropy code decoding unit 200, an inverse quantization / inverse transform unit 210, a synthesis unit 220, a memory 230, and a prediction unit 240.

[0102] The entropy code decoding unit 200 decodes the encoded stream (bitstream) and outputs the quantization conversion coefficients corresponding to the decoded block to the inverse quantization / inverse conversion unit 210. The entropy code decoding unit 200 also obtains flag information related to prediction processing from the encoded stream and outputs the flag information related to prediction processing to the prediction unit 240.

[0103] Here, the entropy code decoding unit 200 may obtain flag information from the encoded stream indicating whether or not intra prediction has been applied for each block to be decoded. The entropy code decoding unit 200 may also obtain flag information from the encoded stream indicating whether or not intra curve prediction has been applied for each block to be decoded to which intra prediction has been applied.

[0104] The entropy code decoding unit 200 may, for a decoded block to be decoded to which intra prediction has been applied, obtain from the encoded stream flag information indicating the intra prediction mode of the decoded block, rather than obtaining from the encoded stream flag information indicating the sub-prediction mode of each sub-region subBlock sub-prediction mode subPredMode.

[0105] The inverse quantization / inverse transformation unit 210 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization / inverse transformation unit 210 comprises an inverse quantization unit 211 and an inverse transformation unit 212.

[0106] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 recovers the conversion coefficients of the decoded block by inverse quantization of the quantization conversion coefficients output from the entropy code decoding unit 200 using the quantization parameter (Qp) and the quantization matrix, and outputs the recovered conversion coefficients to the inverse conversion unit 212.

[0107] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output from the inverse quantization unit 211 to restore the predicted residuals, and outputs the restored predicted residuals (restored predicted residuals) to the synthesis unit 220.

[0108] The synthesis unit 220 decodes (reconstructs) the original block by combining the predicted residual output from the inverse transform unit 212 and the predicted block output from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded block to the memory 230.

[0109] Memory 230 stores the decoded blocks output from the synthesis unit 220 and accumulates the decoded blocks as decoded images on a frame-by-frame basis. Memory 230 outputs the decoded blocks or decoded images to the prediction unit 240. Memory 230 also outputs the decoded images on a frame-by-frame basis to the outside of the decoding device 2. A loop filter may be provided between the synthesis unit 220 and memory 230.

[0110] The prediction unit 240 performs predictions on a block-by-block basis. The prediction unit 240 includes an inter-prediction unit 241, an intra-prediction unit 242, and a switching unit 243.

[0111] The interpretation unit 241 performs interpretation using correlations between frames. Specifically, the interpretation unit 241 predicts the blocks to be decoded using the decoded images stored in the memory 230 as reference images based on interpretation information (e.g., motion vector information) output from the entropy code decoding unit 200, generates interpretation prediction blocks, and outputs the generated interpretation prediction blocks to the switching unit 243.

[0112] The intra-prediction unit 242 performs intra-prediction using spatial correlations within the frame. Specifically, the intra-prediction unit 242 uses an intra-prediction mode corresponding to the intra-prediction flag information output from the entropy code decoding unit 200 to generate an intra-prediction block by referring to the decoded pixels (reference pixels) surrounding the decoded block in the decoded image stored in the memory 230, and outputs the generated intra-prediction block to the switching unit 243. In this embodiment, the intra-prediction unit 242 corresponds to the intra-prediction device on the decoding side.

[0113] The switching unit 243 switches between the inter-prediction block output by the inter-prediction unit 241 and the intra-prediction block output by the intra-prediction unit 242, and outputs one of the prediction blocks to the synthesis unit 220.

[0114] Next, the intra-prediction unit 242 according to this embodiment will be described.

[0115] The intra-prediction unit 242 has a prediction mode identification unit 2421 that identifies the intra-prediction mode predMode to be applied to the block to be decoded based on the flag information acquired by the entropy code decoding unit 200. The intra-prediction unit 242 predicts the block to be decoded using the intra-prediction mode predMode identified by the prediction mode identification unit 2421. In this embodiment, the intra-prediction mode predMode for each block to be decoded is acquired by the entropy code decoding unit 200, but the decoding side may identify the intra-prediction mode predMode based on the parameters of the block to be decoded (position, shape, and prediction mode of adjacent regions).

[0116] The intra-prediction unit 242 according to this embodiment includes an intra-curve prediction unit 2422. Figure 13 is a diagram showing the configuration of the intra-curve prediction unit 2422 according to this embodiment. As shown in Figure 13, the intra-curve prediction unit 2422 includes a region division unit 2422a, a derivation unit 2422b, a region prediction unit 2422c, and a region synthesis unit 2422d.

[0117] The operation of the region division unit 2422a, the derivation unit 2422b, the region prediction unit 2422c, and the region synthesis unit 2422d is the same as the operation of the region division unit 1722a, the derivation unit 1722b, the region prediction unit 1722c, and the region synthesis unit 1722d described above.

[0118] As described above, the decoding device 2 according to this embodiment has an intra-prediction unit 242 (intra-prediction device) that performs intra-prediction on a block-by-block basis that constitutes the image. The intra-prediction unit 242 includes a region division unit 2422a that divides a target block to which an intra-prediction mode having a linear prediction direction is assigned into a plurality of sub-regions, a derivation unit 2422b that derives a sub-prediction mode for each of the plurality of sub-regions according to a reference curve obtained from the linear prediction direction based on the intra-prediction mode, a region prediction unit 2422c that generates a plurality of prediction regions by predicting each of the plurality of sub-regions using the sub-prediction modes, and a region synthesis unit 2422d that synthesizes the plurality of prediction regions and outputs a prediction block for the target block. This makes it possible to predict curved object boundaries with high accuracy and is expected to improve coding efficiency.

[0119] <Operation> Next, the operation according to this embodiment will be described. Here, we will describe the operation of the decoding device 2 that relates to intra-curve prediction.

[0120] Figure 14 shows the operation of the decoding device 2 according to this embodiment. The decoding device 2 performs this operation for each block to be decoded.

[0121] As shown in Figure 14, in step S11, the entropy code decoding unit 200 parses the prediction mode to be applied to the decoded block based on the encoded stream. Specifically, the entropy code decoding unit 200 obtains flag information related to the prediction process from the flag information included in the encoded stream and outputs the obtained flag information to the prediction unit 240. This flag information includes an intra-prediction flag indicating whether or not intra-prediction has been applied, and a curve prediction flag indicating whether or not intra-curve prediction has been applied to the decoded block to which intra-prediction has been applied.

[0122] In step S12, the prediction unit 240 determines whether the intra-prediction flag is "1". If the intra-prediction flag is not "1" (step S12: NO), in step S16, the inter-prediction unit 241 predicts the block to be decoded by inter-prediction and outputs the predicted block.

[0123] If the intra-prediction flag is "1" (step S12: YES), in step S13, the intra-prediction unit 242 determines whether the curve prediction flag is "1". In this case, the intra-prediction unit 242 obtains flag information indicating the intra-prediction mode predMode (for example, flag information indicating the intra-prediction mode number) from the entropy code decoding unit 200. This intra-prediction mode predMode is assumed to be directionality prediction.

[0124] If the curve prediction flag is not "1" (step S13: NO), in step S15, the intra prediction unit 242 predicts the block to be decoded using an intra prediction other than intra curve prediction (for example, intra straight line prediction) based on the intra prediction mode predMode, and outputs the predicted block.

[0125] If the curve prediction flag is "1" (step S13: YES), in step S14, the intra curve prediction unit 2422 of the intra prediction unit 242 predicts the blocks to be decoded by intra curve prediction based on the intra prediction mode predMode, and outputs the predicted blocks.

[0126] Figure 15 shows the operation of the intra-curve prediction unit 2422 according to this embodiment.

[0127] As shown in Figure 15, in step S21, the region division unit 2422a divides the decoded block to which an intra-prediction mode predMode having a linear prediction direction has been assigned into a plurality of sub-regions subBlock_i. Here, the region division unit 2422a may determine the division direction and the number of divisions based on the parameters of the decoded block (position, shape, and prediction mode) and the parameters of adjacent decoded regions. The region division unit 2422a may also obtain information on the division shape and position determined on the encoding side from the entropy code decoding unit 200.

[0128] In step S22, the derivation unit 2422b sets point C0 = (0,0) and point C2 = (w-1,h-1) as shown in Figure 6(a), and sets the intersection point of the line passing through point C0 and parallel to the intra prediction mode predMode of the block to be decoded and the boundary line of the block to be decoded as C1 = (c x ,c y ) The derivation unit 2422b then derives a quadratic Bézier curve using these three points as control points as the reference curve.

[0129] In step S23, the derivation unit 2422b sets each intersection point between the boundary line of each of the multiple sub-regions subBlock_i and the reference curve as a reference point, as shown in Figure 6(b).

[0130] In step S24, the derivation unit 2422b derives the subprediction mode subPredMode for each of the multiple sub-regions subBlock_i based on each straight line obtained by connecting the reference points. Specifically, as shown in Figure 6(c), the derivation unit 2422b derives the subprediction mode subPredMode for each sub-region subBlock_i by selecting the intraprediction mode (see Figure 3) that has the closest slope to the slope of the straight line obtained by connecting the reference points. Note that here the slope of the straight line is the slope between two points, but the slope may also be determined using methods such as differentiation of the curve at the reference points.

[0131] In step S25, the derivation unit 2422b derives reference pixels for predicting the sub-region subBlock using the derived sub-prediction mode subPredMode. The method for expanding the reference pixels is either method 1 or method 2 described above. Then, the region prediction unit 2422c predicts each of the multiple sub-regions subBlock_i using the sub-prediction mode subPredMode and reference pixels derived by the derivation unit 2422b, and generates multiple predicted regions (predicted regions).

[0132] In step S26, the region merging unit 2422d merges the multiple predicted regions generated by the region prediction unit 2422c and outputs a predicted block (intra-predicted block) for the block to be decoded. The region merging unit 2422d may apply filtering to reduce discontinuities at the boundaries of the sub-region subBlock_i, or it may perform region merging by superimposing the predicted regions of the sub-region subBlock_i and performing calculation processing such as taking a weighted average. In this case, the filtering method and calculation method may be determined based on the parameters of the sub-region subBlock_i.

[0133] Figures 16 and 17 show examples of predictions made by the intra-curve prediction unit 2422 according to this embodiment. Here, the target block is divided into four vertical sections. In Figure 16, the intra-prediction mode predMode is mode "24", and in Figure 17, the intra-prediction mode predMode is mode "42". Furthermore, the target block size is assumed to be 32 × 32, the sub-region subBlock_i size is assumed to be 8 × 32 pixels (divided into four vertical sections), and the reference curve is assumed to be a quadratic Bézier curve.

[0134] Figures 16(a) and 17(a) show examples of predictions when intra-prediction (i.e., intra-linear prediction) is performed on a target block using the intra-prediction mode predMode, which has a linear prediction direction. As shown in Figures 16(a) and 17(a), intra-linear prediction is performed by linearly copying a portion of the reference pixels.

[0135] Figures 16(b) and 17(b) show the reference curve and sub-prediction modes in the intra-curve prediction according to this embodiment, compared with the linear prediction direction. As shown in Figures 16(b) and 17(b), the intra-curve prediction unit 2422 derives a sub-prediction mode subPredMode[i] corresponding to the reference curve obtained from the linear prediction direction for each sub-region subBlock_i, predicts each sub-region subBlock_i using the sub-prediction mode subPredMode[i] to generate multiple prediction regions, and outputs a prediction block for the block to be encoded by combining the multiple prediction regions.

[0136] Figures 16(c) and 17(c) show prediction blocks using intra-curve prediction according to this embodiment. As shown in Figures 16(c) and 17(c), prediction blocks using intra-curve prediction perform curvilinear prediction. This makes it possible to predict curvilinear object boundaries with high accuracy, and an improvement in coding efficiency can be expected.

[0137] <Other Embodiments> In the above embodiment, an example was described in which the block to be encoded is divided into regions and predictions are made for each region. However, predictions may also be made line by line. In this case, each line can be considered a sub-region to be predicted.

[0138] In the embodiments described above, an example was given where the reference curve is a Bézier curve, but different curves may be used based on the position, shape, and prediction mode of the block to be encoded.

[0139] In the above embodiment, an example was described in which the prediction residual of the entire target block to be encoded, obtained by synthesizing sub-regions, is calculated and transformed. However, the prediction residual may be calculated and transformed for each sub-region. Specifically, prediction processing may be performed according to the intra-prediction mode determined for each sub-region, and transformation processing may be applied to the prediction residual of each sub-region. Furthermore, by applying quantization, inverse quantization, and inverse transformation to each sub-region, the prediction residual of that sub-region can be restored, and by synthesizing it with the prediction residual of the sub-region, the decoded image of the sub-region can be acquired sequentially. In the prediction processing after the sub-region, the decoded image of the sub-region may be used for the prediction processing of the sub-region.

[0140] In the above embodiment, an example was described in which each sub-prediction mode of a sub-region is derived based on a straight line obtained by connecting each boundary line of the sub-region with each intersection point of the reference curve. However, a reference position may be set for each sub-region, and each sub-prediction mode of the sub-region may be derived according to the slope of the reference curve at this reference position. Figure 18 shows the sub-prediction modes when the reference position is the intersection point of the left side of each sub-region with the reference curve.

[0141] In the example above, the reference position was defined as the intersection point of the left-hand side of each sub-region and the reference curve, but this is not limited to this. For example, the reference curve could be defined as the center position of the sub-region in the x-axis direction, or as the intersection point of the right-hand side of the sub-region and the reference curve. By taking as many reference positions as there are divisions on the reference curve, the required number of sub-prediction modes can be obtained. The slope of the curve at the reference position can be derived by calculating the derivative of the reference curve at the reference position.

[0142] In the above embodiment, the prediction mode of a block may be used as a candidate prediction mode when encoding a block adjacent to the block to be encoded. For example, the prediction mode of the sub-region closest to the adjacent block may be used as a candidate, the average value of multiple prediction modes may be used, or the prediction mode of the block may be used.

[0143] A program may be provided that causes a computer to perform each process performed by the encoding device 1. A program may also be provided that causes a computer to perform each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, but may be a recording medium such as a CD-ROM or DVD-ROM.

[0144] The circuits that perform each process carried out by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chipset, SoC). The circuits that perform each process carried out by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chipset, SoC).

[0145] Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to those described above, and various design changes can be made without departing from the gist of the invention. [Explanation of Symbols]

[0146] 1: Encoding device 2: Decryption device 100: Block division section 110: Subtraction Unit 120: Conversion / Quantization Unit 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse Quantization / Inverse Transformation Unit 141: Inverse quantization section 142: Inverse Transform Section 150: Synthesis part 160: Memory 170: Prediction Department 171: Interpretation Department 172: Intra Prediction Unit 173: Switching section 200: Entropy code decoding unit 210: Inverse Quantization / Inverse Transformation Unit 211: Inverse quantization section 212: Inverse transformation section 220: Synthesis part 230: Memory 240: Prediction Department 241: Interpretation Department 242: Intra Prediction Unit 243: Switching section 1721: Prediction mode determination unit 1722: Intra-curve prediction unit 1722a: Area division part 1722b: Derivation part 1722c: Region prediction unit 1722d: Area synthesis section 2421: Prediction mode identification unit 2422: Intra curve prediction unit 2422a: Area division part 2422b: Derivation part 2422c: Region prediction unit 2422d: Area synthesis section

Claims

1. A prediction device that performs predictions on a block-by-block basis, A region division unit divides the target block to which a prediction mode controlling intra-prediction processing has been assigned into multiple sub-regions, A derivation unit that derives sub-prediction modes for the plurality of sub-regions according to the prediction mode, the block shape of the target block, and the division shape of the sub-region, A region prediction unit that predicts each of the multiple sub-regions using the sub-prediction mode and generates multiple prediction regions, An intra-prediction device comprising: a region merging unit that synthesizes the plurality of prediction regions and outputs a prediction block for the target block.

2. When sequentially predicting each of the aforementioned sub-regions, The derivation unit derives the predicted pixels included in the predicted sub-region as reference pixels, The intra prediction device according to claim 1, characterized in that the region prediction unit predicts an unpredicted sub-region adjacent to the predicted sub-region using the derived reference pixels.

3. When predicting each of the aforementioned sub-regions in parallel, The derivation unit derives reference pixels for each sub-region from the decoded reference pixels surrounding the target block, The intra prediction device according to claim 1, characterized in that the region prediction unit predicts each of the plurality of sub-regions in parallel using the derived reference pixels.

4. An encoding device characterized by comprising an intra-prediction device according to any one of claims 1 to 3.

5. It further comprises an entropy encoding unit that outputs an encoded stream, The encoding apparatus according to claim 4, characterized in that the entropy encoding unit includes flag information indicating the prediction mode in the encoding stream, but does not include flag information indicating the sub-prediction mode in the encoding stream.

6. A decoding device characterized by comprising an intra-prediction device according to any one of claims 1 to 3.

7. It further comprises an entropy code decoding unit for decoding the encoded stream, The decoding device according to claim 6, characterized in that the entropy code decoding unit obtains flag information indicating the prediction mode from the coded stream without obtaining flag information indicating the sub-prediction mode from the coded stream.

8. A program characterized by causing a computer to function as an intra-prediction device according to any one of claims 1 to 3.

Images