Intra prediction device, image encoding device, image decoding device, and program
The intra prediction device addresses the code increase issue by diagonally dividing blocks into non-rectangular regions and determining intra prediction modes based on surrounding blocks, enhancing prediction accuracy and reducing data transmission in video encoding.
Patent Information
- Application Number
- JP2023131193
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-08-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2039-08-05
AI Technical Summary
Intra prediction in video encoding methods, such as VVC, faces an increase in code transmission due to the use of triangulation prediction for triangular areas, which is not currently applicable, despite the effectiveness of predicting triangular patterns and textures.
An intra prediction device that divides prediction target blocks diagonally into non-rectangular regions, determining dividing lines and intra prediction modes based on surrounding decoded blocks, thereby eliminating the need to transmit additional flags and identifiers, allowing triangulation prediction in intra prediction without increasing code transmission.
Enables accurate intra prediction with reduced code transmission by determining intra prediction modes for triangular areas based on surrounding blocks, improving prediction accuracy while minimizing the amount of data transmitted.
Smart Images

Figure 0007590510000001 
Figure 0007590510000002 
Figure 0007590510000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an intra prediction device, an image encoding device, an image decoding device, and a program. [Background technology]
[0002] Research has been conducted into video coding methods to compress the data volume of still images and moving images during transmission and storage. In recent years, the spread of ultra-high resolution video, such as 8K-SHV, has progressed in video coding technology, and coding methods such as AVC / H.264 and HEVC / H.265 are known as methods for transmitting huge amounts of moving image data.
[0003] VVC, a next-generation coding method being standardized jointly by MPEG and ITU, uses intra-prediction that exploits spatial correlation within a frame and inter-prediction that exploits spatial correlation between frames (see Non-Patent Document 1).
[0004] In intra prediction, decoded reference pixels surrounding the target block for intra prediction are used, and the image encoding device selects the optimal mode from a total of 67 prediction modes consisting of planar prediction, DC prediction, and 65 directional prediction modes, and the selected information is transmitted to the image decoding device.
[0005] In VVC, when generating a predicted image using inter prediction, a technology is implemented in which a target block for inter prediction is divided diagonally and a separate motion prediction is used for each triangular region obtained by the division to improve prediction accuracy. At that time, a flag indicating whether or not to use prediction for each triangular region (hereinafter referred to as "triangular division prediction") and a flag indicating which of the two diagonals to use for division are transmitted from the image encoding device to the image decoding device. [Prior art documents] [Non-patent literature]
[0006] [Non-Patent Document 1] JVET-N1001 Versatile Video Coding (Draft 5) DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]
[0007] Although triangular decomposition prediction is possible in inter prediction and is not used in intra prediction, pictures and textures are often triangular in shape even in intra prediction, and it is considered effective to perform prediction for each triangular region. However, since intra prediction has a large number of modes, there is a drawback in that the amount of code to be transmitted increases when the intra prediction mode for each triangular region is transmitted from the image encoding device to the image decoding device.
[0008] Therefore, an object of the present invention is to provide an intra prediction device, an image encoding device, an image decoding device, and a program that enable triangular decomposition prediction to be used in intra prediction while suppressing an increase in the amount of code to be transmitted. [Means for solving the problem]
[0009] An intra prediction device according to a first aspect is an intra prediction device that performs intra prediction on a block obtained by dividing an image, and is equipped with: a determination unit that, when dividing a prediction target block for intra prediction into a plurality of non-rectangular regions, determines a division line to be used for the division and determines an intra prediction mode to be applied to each of the plurality of non-rectangular regions; and a prediction unit that generates a prediction image for each of the plurality of non-rectangular regions using the intra prediction mode determined by the determination unit, synthesizes the generated prediction images, and outputs an intra prediction image, and the determination unit determines at least one of the division line and the intra prediction mode to be applied to each of the plurality of non-rectangular regions based on the intra prediction mode applied to decoded blocks surrounding the prediction target block. The intra prediction device according to the embodiment includes a determination unit that, when a prediction target block for intra prediction is divided by a diagonal, determines a division diagonal to be used for the division from among two diagonals of the prediction target block, and determines an intra prediction mode to be applied to each of a plurality of triangular regions obtained by dividing the prediction target block, and a prediction unit that generates a prediction image for each of the plurality of triangular regions using the intra prediction mode determined by the determination unit, synthesizes the generated prediction images, and outputs an intra prediction image. The determination unit determines at least one of the division diagonal and the intra prediction mode to be applied to each of the plurality of triangular regions based on the intra prediction mode applied to the decoded blocks surrounding the prediction target block.
[0010] An image encoding device according to a second aspect includes the intra prediction device according to the first aspect.
[0011] An image decoding device according to a third aspect includes the intra prediction device according to the first aspect.
[0012] A program according to a fourth aspect causes a computer to function as the intra prediction device according to the first aspect. Effect of the Invention
[0013] According to the present invention, it is possible to provide an intra prediction device, an image encoding device, an image decoding device, and a program that enable triangulation prediction to be used in intra prediction while suppressing an increase in the amount of code to be transmitted. [Brief description of the drawings]
[0014] [Figure 1] FIG. 1 is a diagram showing a configuration of an image encoding device according to an embodiment. [Diagram 2] FIG. 11 is a diagram showing intra prediction mode candidates according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating triangulation prediction according to an embodiment. [Figure 4] FIG. 2 is a diagram showing a configuration of an intra-prediction unit of an image encoding device according to an embodiment. [Diagram 5] 11A to 11C are diagrams illustrating a specific example of a process performed by a determination unit according to the embodiment. [Figure 6] FIG. 2 is a diagram showing a configuration of an image decoding device according to an embodiment. [Figure 7] FIG. 2 is a diagram showing a configuration of an intra-prediction unit of an image decoding device according to an embodiment. [Figure 8] FIG. 11 is a diagram illustrating an example of an operation flow of an intra prediction unit according to the embodiment. [Figure 9] 13A and 13B are diagrams illustrating an example of processing by a determination unit according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] An image encoding device and an image decoding device according to an embodiment will be described with reference to the drawings. The image encoding device and the image decoding device according to the embodiment respectively encode and decode moving images as represented by MPEG. In the following description of the drawings, the same or similar parts are referred to as the same or similar parts. are similarly numbered.
[0016] <Configuration of the image encoding device> First, the configuration of an image coding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an image coding device 1 according to this embodiment.
[0017] As shown in FIG. 1, the image encoding device 1 includes a block division unit 100, a subtraction unit 110, a transformation and quantization unit 120, an entropy encoding unit 130, an inverse quantization and inverse transform unit 140, a synthesis unit 150, a memory 160, and a prediction unit 170.
[0018] The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) constituting a moving image, into a plurality of image blocks, and outputs the image blocks obtained by division to the subtraction unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, but may be a rectangle (non-square). The image block is a unit for encoding by the image encoding device 1 (i.e., a block to be encoded) and a unit for decoding by the image decoding device (i.e., a block to be decoded). Such an image block may be called a CU (Coding Unit).
[0019] The subtraction unit 110 calculates a prediction residual representing a difference (error) between a current block to be coded output from the block division unit 100 and a prediction block obtained by predicting the current block to be coded by the prediction unit 170. Specifically, the subtraction unit 110 calculates the prediction residual by subtracting each pixel value of the prediction block from each pixel value of the block, and outputs the calculated prediction residual to the transformation / quantization unit 120.
[0020] The transform / quantization unit 120 performs orthogonal transform processing and quantization processing on a block-by-block basis. The transform / quantization unit 120 includes a transform unit 121 and a quantization unit 122.
[0021] The transform unit 121 performs an orthogonal transform process on the prediction residual output from the subtraction unit 110 to calculate an orthogonal transform coefficient, and outputs the calculated orthogonal transform coefficient to the quantization unit 122. The orthogonal transform refers to, for example, a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), or the like.
[0022] The quantization unit 122 quantizes the orthogonal transform coefficients output from the transformation unit 121 using a quantization parameter (Qp) and a quantization matrix, and outputs the quantized orthogonal transform coefficients to the entropy coding unit 130 and the inverse quantization / inverse transform unit 140. Note that the quantization parameter (Qp) is a parameter commonly applied to each orthogonal transform coefficient in a block, and is a parameter that determines the coarseness of quantization. The quantization matrix is a matrix whose elements are quantization values used when quantizing each orthogonal transform coefficient.
[0023] The entropy coding unit 130 performs entropy coding on the orthogonal transform coefficients output from the quantization unit 122, performs data compression to generate coded data (bit stream), and outputs the coded data to the outside of the image coding device 1. For the entropy coding, Huffman codes, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used. Note that information such as a flag and an identifier related to prediction is input from the prediction unit 170 to the entropy coding unit 130, and the entropy coding is also performed on the input information.
[0024] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 140 includes an inverse quantization unit 141 and an inverse transform unit 142.
[0025] The inverse quantization unit 141 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 122. Specifically, the inverse quantization unit 141 restores the orthogonal transform coefficients by inverse quantizing the orthogonal transform coefficients output from the quantization unit 122 using a quantization parameter (Qp) and a quantization matrix, and outputs the restored orthogonal transform coefficients to the inverse transform unit 142.
[0026] The inverse transform unit 142 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121. For example, when the transform unit 121 performs discrete cosine transform, the inverse transform unit 142 performs inverse discrete cosine transform. The inverse transform unit 142 performs inverse orthogonal transform processing on the orthogonal transform coefficients output from the inverse quantization unit 141 to restore prediction residuals, and outputs restored prediction residuals, which are the restored prediction residuals, to the synthesis unit 150.
[0027] The synthesis unit 150 synthesizes, on a pixel-by-pixel basis, the reconstructed prediction residual output from the inverse transform unit 142 with the predicted block output from the prediction unit 170. The synthesis unit 150 adds each pixel value of the reconstructed prediction residual to each pixel value of the predicted block to decode (reconstruct) the block to be coded, and outputs the decoded block (reconstructed block) to the memory 160.
[0028] The memory 160 stores the decoded blocks output from the synthesis unit 150, and accumulates the decoded blocks as decoded images on a frame-by-frame basis. The memory 160 outputs the stored decoded blocks or decoded images to the prediction unit 170. Note that a loop filter may be provided between the synthesis unit 150 and the memory 160.
[0029] The prediction unit 170 performs prediction on a block-by-block basis. The prediction unit 170 includes an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173. In this embodiment, the intra prediction unit 172 corresponds to an intra prediction device provided in the image encoding device 1.
[0030] The inter prediction unit 171 uses the decoded image stored in the memory 160 as a reference image to calculate a motion vector using a technique such as block matching, predicts the block to be encoded to generate an inter prediction block, and outputs the generated inter prediction block to the switching unit 173.
[0031] Here, the inter prediction unit 171 selects an optimal inter prediction method from among inter prediction using multiple reference images (typically bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130.
[0032] The intra prediction unit 172 generates an intra prediction block by referring to decoded pixel values in the decoded image stored in the memory 160 that are located around the block to be encoded, and outputs the generated intra prediction block to the switching unit 173.
[0033] In general, the intra prediction unit 172 selects a prediction mode to be applied to a prediction target block of intra prediction from among a plurality of intra prediction modes, and predicts the target block of intra prediction using the selected prediction mode. The intra prediction unit 172 outputs an identifier related to the selected prediction mode to the entropy coding unit 130.
[0034] FIG. 2 is a diagram showing candidates of intra prediction modes according to the present embodiment. Here, candidates of intra prediction modes used for luminance blocks of the current block to be coded are shown. As shown in FIG. 2, there are 67 intra prediction mode candidates, from 0 to 66. The prediction mode mode "0" is planar prediction, the prediction mode mode "1" is DC prediction, and the prediction mode modes "2" to "66" are directional prediction. In directional prediction, the direction of the arrow indicates the reference direction, the starting point of the arrow indicates the position of the pixel to be predicted, and the end point of the arrow indicates the position of the reference pixel used in predicting the pixel to be predicted. There are mode "2", which is a prediction mode that references the lower left direction, and mode "66", which is a prediction mode that references the upper right direction, as reference directions parallel to the diagonal line passing through the upper right vertex and the lower left vertex of the block, and mode numbers are assigned at predetermined angles from mode "2" to mode "66" in a clockwise direction.
[0035] The switching unit 173 switches between the inter predicted block output from the inter prediction unit 171 and the intra predicted block output from the intra prediction unit 172, and outputs either the predicted block to the subtraction unit 110 and the synthesis unit 150.
[0036] In the image encoding device 1 configured in this manner, the intra prediction unit 172 is capable of performing triangular partitioning prediction in which a prediction target block for intra prediction is divided along a diagonal line and a different intra prediction mode is applied to each of the triangular regions obtained by the division. For example, the intra prediction unit 172 applies triangular partitioning prediction to a prediction target block that constitutes a picture or texture having a triangular shape.
[0037] 3 is a diagram showing triangular decomposition prediction according to this embodiment. Note that, although an example in which the block shape of the prediction target block is a square will be mainly described below, the block shape of the prediction target block may be a non-square (rectangle).
[0038] As shown in Fig. 3, the intra prediction unit 172 determines one diagonal selected from two diagonals in the prediction target block as a division diagonal. Fig. 3(a) shows an example in which a diagonal passing through the upper left vertex and the lower right vertex of the prediction target block is determined as a division diagonal. Fig. 3(b) shows an example in which a diagonal passing through the upper right vertex and the lower left vertex of the prediction target block is determined as a division diagonal.
[0039] In the example shown in Figure 3(a), the intra prediction unit 172 divides the prediction target block into two, an upper right triangular region (Partition 1) and a lower left triangular region (Partition 2), along a diagonal line passing through the upper left vertex and the lower right vertex, generates two predicted images corresponding to the two divided regions, and outputs an intra prediction block by combining the two predicted images.
[0040] In the example shown in Figure 3 (b), the intra prediction unit 172 divides the prediction target block into two, an upper left triangular region (Partition 1) and a lower right triangular region (Partition 2), along a diagonal line passing through the upper right vertex and the lower left vertex, generates two predicted images corresponding to the two divided regions, and outputs an intra prediction block by combining the two predicted images.
[0041] This triangular decomposition prediction enables fine-grained intra prediction according to the image pattern, thereby improving the prediction accuracy of intra prediction. However, when performing this triangular decomposition prediction, the image encoding device 1 may need to transmit (signal) the following three types of additional information to the image decoding device side.
[0042] 1) "Triangulation prediction flag" indicating whether or not to perform triangulation prediction 2) A "division diagonal identifier" indicating which division diagonal of FIG. 3(a) or FIG. 3(b) is to be applied. 3) An "intra prediction mode identifier" indicating the intra prediction mode to be applied to each triangular region
[0043] In particular, intra prediction has a large number of modes, so intra prediction is performed for each triangular region. If the prediction mode identifier is transmitted from the image encoding device 1, the amount of code to be transmitted increases, which may result in a decrease in encoding efficiency.
[0044] Therefore, in this embodiment, the intra prediction unit 172 determines at least one of the division diagonal and the intra prediction mode to be applied to each of the multiple triangular regions based on the intra prediction mode applied to the decoded blocks surrounding the prediction target block.
[0045] Here, the intra prediction modes applied to the decoded blocks surrounding the prediction target block are information available to both the image encoding device 1 and the image decoding device. By determining the division diagonal based on such information, the need to transmit the division diagonal identifier is eliminated, and by determining the intra prediction mode to be applied to each of the multiple triangular regions based on such information, the need to transmit the intra prediction mode identifier is eliminated.
[0046] Therefore, even when triangular partitioning prediction is used for intra prediction, it is possible to eliminate the need to transmit at least one of the "diagonal partitioning identifier" and the "intra prediction mode identifier", thereby improving the accuracy of intra prediction while suppressing an increase in the amount of code to be transmitted.
[0047] Below, we describe an example in which the intra prediction unit 172 determines both the division diagonal and the intra prediction mode to be applied to each of the multiple triangular regions based on the intra prediction mode applied to the decoded blocks surrounding the block to be predicted.
[0048] However, the intra prediction unit 172 may determine only the division diagonal based on the intra prediction mode applied to the decoded blocks in the periphery of the prediction target block, and may determine the intra prediction mode to be applied to each of the plurality of triangular regions based on another criterion. Alternatively, the intra prediction unit 172 may determine only the intra prediction mode to be applied to each of the plurality of triangular regions based on the intra prediction mode applied to the decoded blocks in the periphery of the prediction target block, and may determine the division diagonal based on another criterion.
[0049] Next, the configuration of the intra prediction unit 172 of the image encoding device 1 according to this embodiment will be described.
[0050] As shown in FIG. 4, the intra prediction unit 172 includes a determination unit 1721 and a prediction unit 1722.
[0051] When dividing a prediction target block for intra prediction by a diagonal, the determining unit 1721 determines a division diagonal to be used for division from among two diagonals of the prediction target block, and determines an intra prediction mode to be applied to each of a plurality of triangular regions obtained by dividing the prediction target block. In this embodiment, the determining unit 1721 determines the division diagonal and the intra prediction mode to be applied to each of a plurality of triangular regions based on the intra prediction mode applied to the decoded blocks surrounding the prediction target block.
[0052] Here, if the functions of the determination unit 1721 are divided into three, namely, (a) "input", (b) "output", and (c) "processing", the results are as follows.
[0053] (A) Input: The determination unit 1721 receives, from the memory 160, intra-prediction mode information used in the surrounding blocks (top and left).
[0054] (i) Output: The determination unit 1721 outputs the division diagonal (shape information of the triangular region) and information indicating the intra prediction mode to be applied to each triangular region.
[0055] (c) Processing: When triangular partitioning prediction is used in intra prediction, the determination unit 1721 determines the partitioning diagonal (shape of the triangular area) and the intra prediction mode to be applied to each triangular area based on the intra prediction modes used in the surrounding blocks (top and left).
[0056] Fig. 5 is a diagram showing a specific example of the process of the determination unit 1721 according to the present embodiment. In Fig. 5, "A" and "B" are directional prediction modes among intra prediction modes, specifically, prediction modes between mode "2" and mode "66". Meanwhile, "C" and "D" are non-directional prediction modes among intra prediction modes, specifically, planar prediction mode which is mode "0", and DC prediction mode which is mode "1". Here, it is assumed that "C" is the planar mode and "D" is the DC mode.
[0057] As shown in Fig. 5, the determination unit 1721 makes a determination based on an intra prediction mode applied to an upper block, which is a decoded block above the prediction target block, and an intra prediction mode applied to a left block, which is a decoded block to the left of the prediction target block. Since the image encoding device 1 performs encoding processing in raster scan order, it is intended to use the upper block and the left block. The determination unit 1721 obtains information on the intra prediction modes of these surrounding blocks from the memory 160.
[0058] In addition, in FIG. 5, an example is shown in which the upper block and the left block are each a square, but at least one of the upper block and the left block may be a non-square. Also, an example is shown in which the size of each of the upper block and the left block is the same as the size of the encoding target block, but at least one of the width of the upper block and the height of the left block may be smaller than the width and height of the encoding target block. When it is assumed that the size of the upper block and the left block is smaller than the size of the encoding target block, for example, the upper block may be an upper adjacent block including a pixel adjacent to the upper right pixel in the prediction target block, and the left block may be a left adjacent block including a pixel adjacent to the lower left pixel in the prediction target block.
[0059] Alternatively, the upper block may be an upper adjacent block including pixels adjacent to the upper leftmost pixel in the prediction target block, and the left block may be a left adjacent block including pixels adjacent to the upper leftmost pixel in the prediction target block.
[0060] Alternatively, the upper block may be the upper adjacent block that has the longest adjacent length among the blocks adjacent to the upper side of the block to be predicted, and the left block may be the left adjacent block that has the longest adjacent length among the blocks adjacent to the left side of the block to be predicted.
[0061] 5(a) to 5(c), when the intra prediction mode applied to the upper block and the intra prediction mode applied to the left block are different from each other, the determining unit 1721 determines a diagonal passing through the upper left vertex and the lower right vertex of the prediction target block as a dividing diagonal. Then, the determining unit 1721 divides the prediction target block into an upper right triangular region adjacent to the upper block and a lower left triangular region adjacent to the left block for handling.
[0062] In the example of FIG. 5(a), the top block has directional prediction mode "A" and the left block has directional prediction mode "B", so the intra prediction modes are different. In the example of FIG. 5(b), the top block has non-directional prediction mode "C" and the left block has directional prediction mode "B", so the intra prediction modes are different. In the example of FIG. 5(c), the top block has non-directional prediction mode "C" and the left block has non-directional prediction mode "D", so the intra prediction modes are different.
[0063] As shown in Figures 5 (a) to (c), the determination unit 1721 determines the intra prediction mode applied to the upper block as the intra prediction mode to be applied to the upper right triangular region, and determines the intra prediction mode of the left block as the intra prediction mode to be applied to the lower left triangular region.
[0064] Specifically, in the example of FIG. 5(a), the determination unit 1721 determines that the directional prediction mode "A" of the top block is applied to the upper right triangular region, and that the directional prediction mode "B" of the left block is applied to the lower left triangular region. In the example of FIG. 5(b), the determination unit 1721 determines that the non-directional prediction mode "C" of the top block is applied to the upper right triangular region, and that the directional prediction mode "B" of the left block is applied to the lower left triangular region. In the example of FIG. 5(c), the determination unit 1721 determines that the non-directional prediction mode "C" of the top block is applied to the upper right triangular region, and that the non-directional prediction mode "D" of the left block is applied to the lower left triangular region.
[0065] In this way, by determining the intra prediction mode applied to the upper block as the intra prediction mode to apply to the upper right triangular region, and determining the intra prediction mode of the left block as the intra prediction mode to apply to the lower left triangular region, the intra prediction mode of adjacent blocks that are highly spatially correlated can be applied to each triangular region, thereby improving the accuracy of intra prediction.
[0066] On the other hand, as shown in Figures 5(d) and (e), when the intra prediction mode applied to the upper block and the intra prediction mode applied to the left block are the same, the determining unit 1721 determines the diagonal passing through the upper right vertex and the lower left vertex of the prediction target block as the dividing diagonal. Then, the determining unit 1721 divides the prediction target block into an upper left triangular region adjacent to both the upper block and the left block, and a lower right triangular region adjacent to neither the upper block nor the left block.
[0067] In the example of Fig. 5(d), the upper block is in directional prediction mode "A" and the left block is also in directional prediction mode "A", so the intra prediction modes are the same. In the example of Fig. 5(e), the upper block is in non-directional prediction mode "C" and the left block is also in non-directional prediction mode "C", so the intra prediction modes are the same.
[0068] The determination unit 1721 determines the same intra prediction mode applied to the upper block and the left block as the intra prediction mode to be applied to the upper left triangular region, as shown in Figures 5(d) and (e). In the example of Figure 5(d), the determination unit 1721 determines to apply directional prediction mode "A" to the upper left triangular region. In the example of Figure 5(e), the determination unit 1721 determines to apply non-directional prediction mode "C" to the upper left triangular region.
[0069] When the same intra prediction mode is applied to the upper block and the left block, the upper block and the left block are highly correlated, and the upper left triangular region between the upper block and the left block is also considered to be highly correlated. Therefore, by applying the same intra prediction mode applied to the upper block and the left block to the upper left triangular region, the intra prediction mode of the adjacent block that is highly spatially correlated can be applied to the upper left triangular region, thereby improving the accuracy of intra prediction.
[0070] Furthermore, as shown in Figures 5(d) and (e), the determining unit 1721 determines a non-directional intra prediction mode as the intra prediction mode to be applied to the lower right triangular region. Specifically, as shown in Figure 5(d), when the same intra prediction mode applied to the upper block and the left block is a directional prediction mode, the determining unit 1721 determines to apply a predetermined non-directional prediction mode "C" to the lower right triangular region.
[0071] When the diagonal passing through the upper right vertex and the lower left vertex of the prediction target block is used as the dividing diagonal, the lower right triangular region is considered to have a low correlation with the intra prediction modes of the blocks above and to the left, and therefore a predetermined non-directional prediction mode is applied to the lower right triangular region.
[0072] On the other hand, when the same intra prediction mode applied to the upper block and the left block is the first non-directional intra prediction mode "C", as shown in Figure 5 (e), the determination unit 1721 determines a second non-directional intra prediction mode "D" different from the first non-directional intra prediction mode "C" as the intra prediction mode to be applied to the lower right triangular region.
[0073] When the diagonal passing through the upper right vertex and the lower left vertex of the prediction target block is used as the dividing diagonal, the lower right triangular region is considered to have a low correlation with the intra prediction modes of the upper and left blocks, and therefore a non-directional prediction mode different from the non-directional prediction modes applied to the upper and left blocks is applied to the lower right triangular region.
[0074] In this way, when triangular partitioning prediction is used in intra prediction, the determination unit 1721 determines the partitioning diagonal (the shape of the triangular area) and the intra prediction mode to be applied to each triangular area based on the intra prediction modes used in the surrounding blocks (top and left), and outputs information indicating the determination to the prediction unit 1722.
[0075] The prediction unit 1722 generates a prediction image for each of the plurality of triangular regions using the intra prediction mode determined by the determination unit 1721, and synthesizes the generated prediction images to output an intra prediction image. The prediction unit 1722 includes a prediction image generation unit 1722a and a prediction image synthesis unit 1722b.
[0076] The functions of the predicted image generating unit 1722a can be divided into three sections, "input," "output," and "processing," as follows.
[0077] Input: The predicted image generating unit 1722a receives information on the division diagonal (shape of the triangular region) and information on the intra prediction mode to be applied to each triangular region from the determining unit 1721. In addition, the predicted image generating unit 1722a receives reference pixel information to be used for intra prediction (i.e., decoded reference pixel information around the prediction target block) from the memory 160.
[0078] Output: The predicted image generator 1722a outputs a predicted image for each triangular region.
[0079] Processing: Based on the information on the division diagonal and the information on the intra prediction mode applied to each triangular region, intra prediction is performed for each triangular region, and a predicted image for each triangular region is generated.
[0080] Moreover, the functions of the predicted image synthesis unit 1722b can be divided into three, namely, "input", "output" and "processing", as follows.
[0081] Input: The predicted image synthesis unit 1722b receives two predicted images corresponding to the two triangular regions from the predicted image generation unit 1722a.
[0082] Output: The predicted image synthesis unit 1722b outputs the synthesized intra-predicted block.
[0083] Processing: The predicted image synthesis unit 1722b synthesizes two predicted images to generate one intra-predicted block. When performing this synthesis, the predicted image synthesis unit 1722b may apply a filter process to the diagonal portion.
[0084] <Configuration of the image decoding device> Next, a configuration of an image decoding device according to this embodiment will be described below. Fig. 6 is a diagram showing the configuration of an image decoding device 2 according to this embodiment.
[0085] As shown in FIG. 6, the image decoding device 2 includes an entropy decoding unit 200 , an inverse quantization and inverse transform unit 210 , a synthesis unit 220 , a memory 230 , and a prediction unit 240 .
[0086] The entropy decoding unit 200 decodes the encoded data generated by the image encoding device 1, and outputs the quantized orthogonal transform coefficients to the inverse quantization / inverse transform unit 210. In addition, the entropy decoding unit 200 obtains an identifier related to prediction (intra prediction and inter prediction), and outputs the obtained identifier to the prediction unit 240.
[0087] In this embodiment, the entropy decoding unit 200 acquires a triangular partitioning prediction flag indicating that triangular partitioning prediction has been applied to a block to which the image encoding device 1 has applied triangular partitioning prediction, and outputs the acquired triangular partitioning prediction flag to the prediction unit 240 (intra prediction unit 242).
[0088] The inverse quantization and inverse transform unit 210 performs inverse quantization processing and inverse orthogonal transform processing on a block-by-block basis. The inverse quantization and inverse transform unit 210 includes an inverse quantization unit 211 and an inverse transform unit 212.
[0089] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the image encoding device 1. The inverse quantization unit 211 reconstructs the orthogonal transform coefficients of the block to be decoded by inverse quantizing the quantized orthogonal transform coefficients output from the entropy decoding unit 200 using a quantization parameter (Qp) and a quantization matrix, and outputs the reconstructed orthogonal transform coefficients to the inverse transform unit 212.
[0090] The inverse transform unit 212 performs inverse orthogonal transform processing corresponding to the orthogonal transform processing performed by the transform unit 121 of the image encoding device 1. The inverse transform unit 212 performs inverse orthogonal transform processing on the orthogonal transform coefficients output from the inverse quantization unit 211 to restore the prediction residual, and outputs the restored prediction residual (restored prediction residual) to the synthesis unit 220.
[0091] The synthesis unit 220 decodes (reconstructs) the original block by synthesizing the prediction residual output from the inverse transform unit 212 and the prediction block output from the prediction unit 240 on a pixel-by-pixel basis, and outputs the decoded block to the memory 230.
[0092] The memory 230 stores the decoded blocks output from the synthesis unit 220, and accumulates the decoded blocks as decoded images on a frame-by-frame basis. The memory 230 outputs the decoded blocks or the decoded images to the prediction unit 240. The memory 230 also outputs the decoded images on a frame-by-frame basis to the outside of the image decoding device 2. Note that a loop filter may be provided between the synthesis unit 220 and the memory 230.
[0093] The prediction unit 240 performs prediction on a block-by-block basis. The prediction unit 240 has an inter prediction unit 241, an intra prediction unit 242, and a switching unit 243. In this embodiment, the intra prediction unit 242 corresponds to an intra prediction device provided in the image decoding device 2.
[0094] The inter prediction unit 241 predicts a current block to be decoded by inter prediction using a decoded image stored in the memory 230 as a reference image. The inter prediction unit 241 generates an inter prediction block by performing inter prediction according to an identifier, a motion vector, and the like output from the entropy decoding unit 200, and outputs the generated inter prediction block to the switching unit 243.
[0095] The intra prediction unit 242 generates an intra prediction block by referring to decoded pixel values around a prediction target block for intra prediction (a decoding target block) among the decoded images stored in the memory 160, and outputs the generated intra prediction block to the switching unit 243. The intra prediction unit 242 determines whether or not to apply triangular partition prediction to the decoding target block based on a triangular partition prediction flag output from the entropy decoding unit 200.
[0096] When triangular decomposition prediction is not applied to the block to be decoded, the intra prediction unit 242 determines the intra prediction mode to be applied to the block to be decoded based on the intra prediction mode identifier output from the entropy decoding unit 200, and outputs an intra prediction block by performing intra prediction using the determined intra prediction mode.
[0097] On the other hand, when applying triangular partition prediction to the block to be decoded, the intra prediction unit 242 determines a partition diagonal and an intra prediction mode to be applied to each of the multiple triangular regions based on the intra prediction mode applied to the decoded blocks surrounding the block to be predicted. The intra prediction unit 242 then divides the block to be predicted into two along the determined partition diagonal, generates two prediction images corresponding to the two partition regions using the determined intra prediction mode, and outputs an intra prediction block by synthesizing the two prediction images.
[0098] The switching unit 243 switches between the inter-predicted block output from the inter-prediction unit 241 and the intra-predicted block output from the intra-prediction unit 242 , and outputs either the predicted block to the synthesis unit 220 .
[0099] 7 is a diagram showing the configuration of the intra prediction unit 242 of the image decoding device 2 according to this embodiment. The intra prediction unit 242 corresponds to the intra prediction device provided in the image decoding device 2.
[0100] 7, the intra prediction unit 242 has a determination unit 2421 and a prediction unit 2422. The prediction unit 2422 also has a prediction image generation unit 2422a and a prediction image synthesis unit 2422b.
[0101] The decision unit 2421, the predicted image generation unit 2422a, and the predicted image synthesis unit 2422b perform the same operations as the decision unit 1721, the predicted image generation unit 1722a, and the predicted image synthesis unit 1722b shown in Fig. 4. However, the decision unit 2421 receives a triangular partitioning prediction flag indicating whether or not to apply triangular partitioning prediction to a block to be decoded in intra prediction from the entropy decoding unit 200, and decides whether or not to apply triangular partitioning prediction based on this triangular partitioning prediction flag.
[0102] <Example of intra prediction operation flow> Next, an example of an operation flow of intra prediction according to this embodiment will be described. Although the operation of intra prediction is the same in the image encoding device 1 and the image decoding device 2, here, the operation of intra prediction (intra prediction unit 242) in the image decoding device 2 will be described.
[0103] Fig. 8 is a diagram showing an example of an operation flow of the intra prediction unit 242. The intra prediction unit 242 performs the operation shown in Fig. 8 when a triangular partition prediction flag indicating that triangular partition prediction is applied to a decoding target block in intra prediction is input from the entropy decoding unit 200.
[0104] As shown in FIG. 8, in step S1, the determination unit 2421 refers to the memory 230 and determines the intra prediction mode applied to the top block and the intra prediction mode applied to the left block. It is determined whether the code is the same as the code.
[0105] If the intra prediction mode applied to the upper block and the intra prediction mode applied to the left block are different from each other (step S1: NO), in step S2, the determination unit 2421 determines a diagonal passing through the upper left vertex and the lower right vertex of the prediction target block as a dividing diagonal, and divides the prediction target block into an upper right triangular region adjacent to the upper block and a lower left triangular region adjacent to the left block (see FIG. 3(a)). In step S3, the determination unit 2421 determines to apply the intra prediction mode applied to the upper block to the upper right triangular region, and to apply the intra prediction mode of the left block to the lower left triangular region.
[0106] On the other hand, if the intra prediction mode applied to the upper block and the intra prediction mode applied to the left block are the same (step S1: YES), in step S6, the determination unit 2421 determines the diagonal passing through the upper right vertex and the lower left vertex of the prediction target block as the dividing diagonal, and divides the prediction target block into an upper left triangular region that is adjacent to both the upper block and the left block, and a lower right triangular region that is not adjacent to either the upper block or the left block (see Figure 3(b)).
[0107] In this case, in step S7, the determining unit 2421 determines whether or not the same intra prediction mode applied to the upper block and the left block is a directional prediction mode.
[0108] If the same intra prediction mode applied to the upper block and the left block is a directional prediction mode (step S7: YES), in step S8, the decision unit 2421 decides to apply the same intra prediction mode applied to the upper block and the left block to the upper left triangular region, and decides to apply a predetermined non-directional prediction mode to the lower right triangular region.
[0109] On the other hand, if the same intra prediction mode applied to the upper block and the left block is a non-directional prediction mode (step S7: NO), in step S9, the determination unit 2421 determines to apply the same intra prediction mode applied to the upper block and the left block to the upper left triangular region, and determines to apply a non-directional intra prediction mode different from the same intra prediction mode (non-directional prediction mode) to the lower right triangular region. For example, if the same intra prediction mode applied to the upper block and the left block is a DC prediction mode, the determination unit 2421 determines to apply a planar prediction mode to the lower right triangular region. Also, if the same intra prediction mode applied to the upper block and the left block is a planar prediction mode, the determination unit 2421 determines to apply a DC prediction mode to the lower right triangular region.
[0110] Then, in step S4, the predicted image generation unit 2422a generates a predicted image of each triangular region using the intra prediction mode determined by the determination unit 2421, while referring to the memory 230. In step S5, the predicted image synthesis unit 2422b synthesizes the predicted images generated by the predicted image generation unit 2422a and outputs an intra predicted image.
[0111] <Other embodiments> In the above-described embodiment, the decision unit 1721 and the decision unit 2421 have determined that a non-directional prediction mode is applied to the lower right triangular region of the block to be predicted when the same intra prediction mode is applied to the upper block and the left block.
[0112] However, in such a case, the determining unit 1721 and the determining unit 2421 may determine a prediction block that is applied to the upper right block, which is a decoded block located to the upper right of the prediction target block, as shown in FIG. The intra prediction mode to be applied to the lower right triangular region is determined based on at least one of the intra prediction mode used in the prediction process and the intra prediction mode applied to the lower left block, which is a decoded block located to the lower left of the block to be predicted.
[0113] 9(a) shows an example in which the same directional prediction mode "A" is applied to the top block and the left block, and FIG. 9(b) shows an example in which the same non-directional prediction mode "C" is applied to the top block and the left block. Note that the top right block is a decoded block adjacent to the right side of the top block, and the bottom left block is a decoded block adjacent to the bottom side of the left block.
[0114] For example, when the same intra prediction mode is applied to the upper right block and the lower left block, the determining unit 1721 and the determining unit 2421 may determine to apply the same intra prediction mode to the lower right triangular region. When different intra prediction modes are applied to the upper right block and the lower left block, the determining unit 1721 and the determining unit 2421 may determine to apply the intra prediction mode applied to the upper right block to the lower right triangular region.
[0115] In the above embodiment, triangular division prediction may also be applied to the upper block or the left block of the prediction target block. In this case, among the adjacent blocks (upper block or left block) to which triangular division prediction is applied, the prediction mode of a triangular area including a side that is in contact with the prediction target block may be regarded as the prediction mode of the adjacent block. Alternatively, the adjacent blocks to which triangular division prediction is applied may be regarded as blocks using a specific prediction mode. Examples of the above-mentioned specific prediction mode include a planar prediction mode and a DC prediction mode.
[0116] A program may be provided that causes a computer to execute each process performed by the image encoding device 1. A program may be provided that causes a computer to execute each process performed by the image decoding device 2. The program may be recorded in a computer-readable medium. Using the computer-readable medium, it is possible to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.
[0117] The image encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC) by integrating circuits that execute the processes performed by the image encoding device 1. The image decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC) by integrating circuits that execute the processes performed by the image decoding device 2.
[0118] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes, etc. are possible without departing from the spirit of the invention. [Explanation of symbols]
[0119] 1: Image encoding device 2: Image decoding device 100: Block division section 110: Subtraction section 120: Transformation and quantization section 121: Conversion section 122: Quantization section 130: Entropy coding unit 140: Inverse quantization and inverse transformation unit 141: Inverse quantization section 142: Inverse conversion section 150: Composite section 160:Memory 170: Prediction Department 171: Inter prediction section 172: Intra prediction unit 173: Switching section 200: Entropy decoding unit 210: Inverse quantization and inverse transformation unit 211: Inverse quantization section 212: Inverse conversion unit 220: Composite section 230: Memory 240: Prediction Department 241: Inter prediction unit 242: Intra prediction unit 243: Switching section 1721: Decision Section 1722: Prediction Department 1722a: Prediction image generation unit 1722b: Prediction image synthesis unit 2421: Decision Section 2422: Prediction Department 2422a: Prediction image generation unit 2422b: Prediction image synthesis unit
Claims
1. An intra prediction device that performs intra prediction on blocks obtained by dividing an image, a determination unit that, when dividing a prediction target block for the intra prediction into a plurality of non-rectangular regions, determines a division line to be used for the division, and determines intra prediction modes different from each other as intra prediction modes to be applied to each of the plurality of non-rectangular regions; a prediction unit that generates a predicted image for each of the non-rectangular regions using the different intra prediction modes determined by the determination unit, combines the generated predicted images, and outputs an intra predicted image corresponding to the block to be predicted; An intra prediction device characterized in that the determination unit determines at least one of the division line and the different intra prediction modes to be applied to each of the multiple non-rectangular regions based on the intra prediction mode applied to decoded blocks surrounding the prediction target block.
2. The intra prediction device as described in Claim 1, characterized in that each of the multiple non-rectangular regions is a non-rectangular region including a triangle.
3. The intra prediction device described in Claim 1, characterized in that the determination unit determines a diagonal direction as the direction of the division line.
4. The intra prediction device according to claim 1, characterized in that the decision unit makes the decision based on an intra prediction mode applied to an upper block, which is a decoded block above the prediction target block, and an intra prediction mode applied to a left block, which is a decoded block to the left of the prediction target block.
5. An image coding device comprising the intra prediction device according to claim 1 .
6. An image decoding device comprising the intra prediction device according to claim 1 .
7. A program for causing a computer to function as the intra prediction device according to claim 1 .
Citation Information
Patent Citations
Image encoding method, image decoding method, image encoder, image decoder, image encoding program, image decoding program, and recording medium recording programs and readable by computer
JP2009284275A
Method and Device for Coding and Decoding Images
US20130301716A1
Methods and apparatus for geometric-based intra prediction
WO2012100047A1
Coding device, decoding device, coding method, and decoding method
WO2018092868A1
