Image encoding device, image encoding method and program, image decoding device, image decoding method and program

Abstract

The present invention provides an image encoding device, an image encoding method, an image decoding device, and an image decoding method that enable control over the encoding of quantization parameters so that subblocks generated by asymmetric partitioning can be compressed with high efficiency. [Solution] The image decoding method by the image decoding device includes the step of decoding an image that is divided into one or more blocks and further composed of one or more subblocks divided from the blocks. The division from the blocks to the subblocks includes asymmetric division. The method also includes the step of decoding quantization parameters in the asymmetrically divided blocks based at least on information regarding the number of pixels in the subblocks.

Description

Technical Field

[0004] , , , ,

[0005] , ,

[0001] The present invention relates to image encoding and decoding.

Background Art

[0002] As an encoding method for compression recording of moving images, the VVC (Versatile Video Coding) encoding method (hereinafter referred to as VVC) is known. In VVC, in order to improve the encoding efficiency, a basic block called CTU (Coding Tree Unit) is divided into rectangular sub-blocks as well as conventional squares. The CTU is up to 128×128 pixels. Japanese Patent Application Laid-Open No. 2012-161074 (Patent Document 1) discloses a technique for calculating the sub-block size for encoding the quantization parameter and enabling the change of the encoding unit of the quantization parameter.

[0003] In recent years, the JVET (Joint Video Experts Team) that standardized VVC has been studying new encoding technologies to achieve encoding efficiency and image quality higher than VVC. As one of the technologies, in order to improve the encoding efficiency, in addition to the conventional division in which the shape of the division generated by one block division process is point-symmetric, the introduction of a division in which the shape of the division is asymmetric is being studied. <000,014><00000,15>In VVC, squares and rectangles are used as the shapes of sub-blocks generated by block division. In addition, the area of the block before division occupying the image is filled without gaps by a combination of square or rectangular sub-blocks by dividing the block into a plurality of sub-blocks. In the sub-blocks after division having such characteristics, a technique for determining whether to encode the quantization parameter based on an index determined based on the number of pixels of the sub-block is adopted.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2012-161074 [Overview of the project] [Problems that the invention aims to solve]

[0006] When using a partition with an asymmetrical shape, multiple subblocks containing subblocks with different pixel counts are generated. Even when the partition shape is asymmetrical, it is required that the data can be compressed efficiently.

[0007] This invention was made to solve the above-mentioned problems, and aims to enable control over the encoding of quantization parameters so that even subblocks generated by asymmetric partitioning can be compressed with high efficiency. [Means for solving the problem]

[0008] In view of the above problems, the present invention provides a method for decoding an image, comprising the steps of: decoding an image that is divided into one or more blocks and further divided into one or more subblocks from the blocks; and the division from the blocks to the subblocks includes an asymmetric division, and decoding quantization parameters in the asymmetrically divided blocks based at least on information regarding the number of pixels in the subblocks. [Effects of the Invention]

[0009] The present invention makes it possible to appropriately encode quantization parameters even in subblocks generated by asymmetric partitioning. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram showing the configuration of the image encoding device in this embodiment. [Figure 2] Block diagram showing the configuration of the image decoding device in this embodiment. [Figure 3]Flowchart showing the image coding process in the image coding device according to this embodiment. [Figure 4] Flowchart showing the image decoding process in the image decoding device according to this embodiment. [Figure 5] Block diagram showing an example of a computer hardware configuration applicable to the image encoding and decoding devices of this embodiment. [Figure 6] Figure showing an example of a bitstream structure. [Figure 7] A diagram showing an example of subblock division used in this embodiment. [Figure 8] This figure shows an example of the number of pixels in the subblock to be processed used in this embodiment. [Figure 9] This figure shows a comparison between the quantization control threshold and the number of pixels in the subblock to be processed in this embodiment. [Figure 10] This figure shows an example of the coding pattern of quantization parameters in subblocks generated using the asymmetric partitioning in this embodiment. [Figure 11] This figure shows the relationship between the encoding of quantization parameters and the significance coefficient in this embodiment. [Figure 12] This figure shows the relationship between the encoding of quantization parameters and the significance coefficient in each asymmetrically divided subblock in this embodiment. [Figure 13] This figure shows the reference for quantization parameters when there is no significant coefficient in the quantization parameter sharing unit in this embodiment. [Figure 14] This figure shows the relationship between the encoding of quantization parameters and the significance coefficient in the ternary tree-divided subblocks in this embodiment. [Figure 15] This figure shows the setting of quantization parameters in subblocks that do not include significance coefficients. [Figure 16] This figure shows the relationship between the encoding of quantization parameters and the significance coefficient in the asymmetrically divided subblocks of a quadtree in this embodiment. [Figure 17] This figure shows an example of an encoding pattern for quantization parameters with the same partition shape as the ternary partition generated using the asymmetric binary tree partition in this embodiment. [Figure 18] A diagram showing an example of an encoding pattern of quantization parameters in the same division shape as the asymmetric quadtree division generated using the asymmetric binary tree division in this embodiment [Figure 19] A diagram showing an example of an encoding pattern of quantization parameters in the same division shape as the asymmetric quadtree division generated using the ternary tree division or the asymmetric binary tree division in this embodiment [Figure 20] A diagram showing another example of an encoding pattern of quantization parameters in sub - blocks generated using the asymmetric division in this embodiment [Figure 21] A diagram showing the relationship between the number of pixels of sub - blocks and the values encoded as quantization control thresholds in this embodiment

Best Mode for Carrying Out the Invention

[0011] Hereinafter, a preferred embodiment will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples.

[0012] <Embodiment 1> FIG. 1 is a block diagram showing an image encoding apparatus according to this embodiment. In FIG. 1, a control unit 100 is a processor that controls the entire image encoding apparatus, and a terminal 101 is an input terminal for inputting image data.

[0013] A block division unit 102 divides an input image into a plurality of basic blocks and outputs an image in units of basic blocks. [[ID=二十九]]

[0014] The generation unit 103 generates and outputs information that serves as a reference for encoding quantization parameters. In this embodiment, the information that serves as a reference for encoding quantization parameters is information that indicates parameters (hereinafter referred to as quantization control thresholds) generated based on the number of pixels in a subblock. The method for generating quantization control thresholds is not limited to one method. The user may input a quantization control threshold, the quantization control threshold may be calculated from the characteristics of the input image, or a pre-specified quantization control threshold may be used as an initial value.

[0015] The prediction unit 104 generates subblocks by dividing the basic block, and performs intra-frame prediction (intra prediction) and inter-frame prediction (inter prediction) on a subblock basis to generate predicted image data. Furthermore, it calculates and outputs the prediction error from the input pixel values ​​(image data) and the predicted image data. In addition, information necessary for prediction, such as subblock division, prediction mode, and motion vector, is also output along with the prediction error. Hereafter, this information necessary for prediction will be referred to as prediction information.

[0016] The transformation and quantization unit 105 performs an orthogonal transformation of the prediction error in subblock units, and then performs quantization to obtain residual coefficients. The quantization parameters are the parameters used to quantize the transformation coefficients obtained by the orthogonal transformation. In this embodiment, the transformation and quantization unit 105 performs both transformation and quantization, but it may also be configured to be divided into a transformation unit that performs transformation and a quantization unit that performs quantization.

[0017] The inverse quantization / inverse transformation unit 106 inversely quantizes the residual coefficients output from the transformation / quantization unit 105 to reconstruct the transformation coefficients, and then performs an inverse orthogonal transformation to reconstruct the prediction error. In this embodiment, the inverse quantization / inverse transformation unit 106 performs inverse quantization and inverse transformation, but it may also be configured to be divided into an inverse quantization unit that performs inverse quantization and an inverse transformation unit that performs inverse transformation.

[0018] Frame memory 108 is memory that stores the replayed image data.

[0019] The image playback unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104, and generates playback image data from this and the input prediction error.

[0020] The in-loop filter unit 109 performs in-loop filtering on the regenerated image, such as deblocking filtering (DBF) and sample adaptive offsetting (SAO). Other in-loop filtering processes include adaptive loop filtering (ALF) and luma mapping with chroma scaling (LMCS).

[0021] The encoding unit 110 encodes the residual coefficients output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104 to generate coded data.

[0022] The integrated encoding unit 111 encodes the information regarding the quantization control threshold from the generation unit 103 to generate header code data. Furthermore, it combines this with the code data output from the encoding unit 110 to form a bitstream.

[0023] Terminal 112 is an output terminal that outputs the bitstream generated by the integrated encoding unit 111 to the outside.

[0024] The image encoding operation in the above-described image encoding device is explained below. In this embodiment, the device is configured to input moving image data frame by frame, but it may also be configured to input still image data for one frame.

[0025] Image data for one frame input from terminal 101 is input to block division unit 102.

[0026] The block division unit 102 divides the input image data into multiple basic blocks and outputs the image of each basic block to the prediction unit 104.

[0027] The prediction unit 104 performs prediction processing on the image data input from the block division unit 102. Specifically, it first determines subblock division, which further divides the basic block into smaller subblocks.

[0028] Figure 7 shows an example of the division types of a basic block. The 700 in the thick-lined frame represents the basic block. For simplicity of explanation, the basic block is assumed to be 32x32 pixels in size, and each rectangle within the thick-lined frame represents a subblock. Figure 7(b) shows an example of a square subblock obtained by division, where the 32x32 pixel basic block is divided into 16x16 pixel subblocks. A division that divides into four equal parts in one step, as in Figure 7(b), is called a quad division. Figures 7(c) to (k) show examples of the types of rectangular subblocks obtained by division. Of these, Figure 7(i) shows a mixture of square and rectangular subblocks. In Figure 7(c), the basic block is divided into a vertically elongated 16x32 pixel rectangular subblock, and in Figure 7(d), it is divided into a horizontally elongated 32x16 pixel rectangular subblock. A division in which the subblocks are symmetrically divided into two, as in Figures 7(c) and (d), is called a binary division. Furthermore, in Figures 7(e) and (f), the image is divided into rectangular subblocks in a ratio of 1:2:1. This type of division, as shown in Figures 7(e) and (f), is called a ternary division. In Figure 7(g), the image is divided into an 8x32 pixel vertical subblock and a 24x32 pixel vertical subblock. In Figure 7(h), the image is divided into a 32x8 pixel horizontal subblock and a 32x24 pixel horizontal subblock. Also, in Figure 7(i), the top left is a 24x8 pixel horizontal subblock, the top right is an 8x8 pixel square subblock, the bottom left is a 24x24 pixel square subblock, and the bottom right is an 8x24 pixel vertical subblock. In Figure 7(j), the image is divided into a 32x20 pixel horizontal subblock and a 32x12 pixel horizontal subblock. Furthermore, Figure 7(k) is divided into four horizontal sub-blocks of 32×4 pixels, 32×16 pixels, 32×8 pixels, and 32×4 pixels.

[0029] In quad partitioning, binary partitioning, and ternary partitioning, the resulting subblocks are symmetrical along a line or at a point. On the other hand, partitioning other than quad partitioning, binary partitioning, and ternary partitioning, as shown in Figures 7(g) to (k), that results in asymmetrical subblocks is called an asymmetric partitioning.

[0030] Furthermore, asymmetric partitioning may also refer to a partitioning method in which a block, when divided into sub-blocks, does not satisfy at least one of the following: horizontal symmetry or vertical symmetry.

[0031] The partition shapes in Figures 7(e) and (f) may be generated by creating three subblocks in a single partition using a ternary partition, or by creating two partitions using an asymmetric binary tree partition. For example, the partition shape in Figure 7(e) can be generated by dividing the basic block into two subblocks using a binary tree partition that divides horizontally in a 3:1 ratio, and then further dividing the larger of the two resulting subblocks horizontally in a 1:2 ratio. Similarly, the partition shape in Figure 7(i) may be generated in a single partition using an asymmetric quadrupine partition, or by creating two partitions using an asymmetric binary tree partition. For example, the partition shape in Figure 7(i) can be generated by first dividing the block into two subblocks using a binary tree partition that divides horizontally in a 3:1 ratio, and then further dividing each of these subblocks vertically in a 1:3 ratio using a binary tree partition. Alternatively, the partition shape shown in Figure 7(i) can be generated by first dividing the block into two subblocks using a binary tree partition that divides vertically in a 1:3 ratio, and then further dividing each of these subblocks horizontally using a binary tree partition in a 3:1 ratio. Furthermore, the partition shape shown in Figure 7(k) can also be generated in a single partition using an asymmetric quadrupine partition. Alternatively, it can be generated by first performing a ternary tree partition that divides in a 1:6:1 ratio, and then performing an asymmetric binary tree partition that further divides the central subblock vertically in a 2:1 ratio. Or, in the first partition, an asymmetric binary tree partition that divides in a 1:7 ratio can be performed, and then, as the second partition, a binary tree partition that divides the larger, lower subblock in a 4:3 ratio can be performed. Furthermore, as the third partition, a binary tree partition that divides the smaller, lower subblock generated in the second partition in a 2:1 ratio can be performed. Note that the partition ratios when performing binary, ternary, and quadrupine partitions on the basic block are not limited to the examples shown in Figure 7. For example, in the case of an asymmetric binary tree partition, a ratio such as 4:3 can be used as long as the number of pixels in the subblock can be represented as an integer. Similarly, in the case of a ternary tree partition, a ratio such as 1:3:1 can be used. This makes it possible to generate subblocks of a wide variety of shapes.

[0032] Thus, in this embodiment, encoding is performed using not only square subblocks but also rectangular subblocks. Furthermore, in this embodiment, information regarding the division type of such basic blocks is encoded as division information. Moreover, in order to obtain a hierarchical structure of subblocks as shown in the left-hand diagram of Figure 9 or Figure 10 described later, the division type information is encoded in a hierarchical manner.

[0033] The prediction unit 104 then determines a prediction mode for each subblock to be processed. Specifically, the prediction unit 104 determines a prediction mode for each subblock, such as intra-prediction using pixels encoded in the same frame as the frame containing each subblock to be processed, or inter-prediction using pixels encoded in different frames. The prediction unit 104 then generates predicted image data from the determined prediction mode and encoded pixels, and further generates a prediction error from the input image data and the predicted image data, which is output to the conversion / quantization unit 105. The prediction unit 104 also outputs information such as subblock division and prediction mode as prediction information to the encoding unit 110 and the image playback unit 107.

[0034] Here, the transformation and quantization processes performed by the transformation / quantization unit 105 will be explained in more detail. The transformation / quantization unit 105 performs frequency transformation on the prediction error of the subblocks that have been predicted by the prediction unit 104, and then performs quantization. Figures 8(a) to (k) show the relationship between the type of block division and the size of the subblocks. The size of the subblock is the number of pixels of the subblock corresponding to the division ratio of the block to be divided, but it may also be represented by an index that substitutes for the number of pixels of the subblock. The generation of the index that substitutes for the number of pixels will be described later. Based on the size of the subblocks to be processed and the quantization control threshold output from the generation unit 103, the transformation / quantization unit 105 decides which subblock units share and encode the quantization parameters. That is, in the subsequent encoding unit 110, it is decided whether or not the quantization parameters should be shared among multiple subblocks according to a comparison between the quantization control threshold and the number of pixels of each subblock. The encoding of the quantization parameters will be described later. There are no particular limitations on how the values ​​of the quantization parameters used for quantization are determined. Users may input the quantization parameters, calculate them from the characteristics of the input image, or use pre-specified initial values.

[0035] Next, we will explain how to determine the unit used to encode the quantization parameters.

[0036] The conversion and quantization unit 105 compares the size of the subblock to be processed with the quantization control threshold to determine in what unit the quantization parameters will be encoded, that is, in what unit the same quantization parameters will be used.

[0037] Here, we will explain how to calculate the size of the subblock using Figure 8.

[0038] Here, the subblock size is determined by the number of pixels in the subblock corresponding to the division ratio of the block being divided. Figure 8 shows examples of subblock division types when dividing a basic block of 32x32 pixels. The numbers listed within each subblock in Figures 8(a) to (k) represent the number of pixels in each subblock. Figure 8(a) shows that the basic block has not been divided at all, and the number of pixels in the subblock is the same as the basic block, 1024 (32x32) pixels. Figure 8(b) shows that the basic block has been quad-divided, and each subblock has 256 pixels. Figures 8(c) and (d) show that the basic block has been binary-divided, and each subblock has 512 pixels. Figures 8(e) and (f) show that the basic block has been ternarily-divided. If the number of pixels in the subblock after division is halved, it will be 512 pixels. On the other hand, if the number of pixels in the subblock after division is quartile, it will be 256 pixels. Furthermore, Figures 8(g) to 8(k) show asymmetrical divisions. For example, Figure 8(g) is divided into a vertical subblock of 256 (8×32) pixels and a vertical subblock of 768 (24×32) pixels, so the division ratio is 1:3. In Figure 8(h), the division ratio is 3:1, and the number of pixels in the left subblock exceeds the number of pixels in the right subblock. Also, in Figure 8(i), the top left is divided into a horizontal subblock of 192 (24×8) pixels, the top right into a square subblock of 64 (8×8) pixels, the bottom left into a square subblock of 576 (24×24) pixels, and the bottom right into a vertical subblock of 192 (8×24) pixels. In Figure 8(j), the upper subblock has 640 (32×20) pixels, and the lower subblock has 384 (32×12) pixels. Figure 8(k) is divided vertically into four sections, with the first subblock from the top having 128 (32×4) pixels, the second subblock having 512 (32×16) pixels, the third subblock having 256 (32×8) pixels, and the fourth subblock having 128 (32×4) pixels. In the asymmetric divisions shown in Figures 8(g) to (k), in the multiple subblocks generated in a single division, there is a subblock with the largest number of pixels and a subblock with the smallest number of pixels.For example, in Figure 8(i), the subblock in the lower left is the subblock with the largest number of pixels, and the subblock in the upper right is the subblock with the smallest number of pixels.

[0039] Here, we will explain how the quantization parameters are encoded according to the quantization control threshold and the number of pixels in the subblocks, using Figure 9. In Figures 9(a) to (e), the square enclosed by the outermost frame represents the basic block. In each of Figures 9(a) to (e), the left side shows the subblock division, and the right side shows the region where the quantization parameters are shared. The numbers within each block in the figures represent the number of pixels in each subblock, which are 256 pixels, 128 pixels, and 64 pixels in descending order of size. Qp represents the quantization parameter. The encoding of quantization parameters when using asymmetric division will be described later using Figure 10. Figures 9(a), (b), (c), (d), and (e) show the cases where the quantization control threshold is 1024 pixels, 512 pixels, 256 pixels, 128 pixels, and 64 pixels, respectively. In the case of Figure 9(a), i.e., when the quantization control threshold is 1024 pixels, a common quantization parameter is used in all subblocks in the figure, and the quantization parameter is encoded in association with the subblock containing the first significance coefficient in the encoding order. In this case, only one quantization parameter is encoded.

[0040] The significance coefficient refers to the non-zero residual coefficients after transformation and quantization. In other words, the presence of a significance coefficient indicates that there is at least one non-zero residual coefficient in the subblock after transformation and quantization. The relationship between the presence or absence of a significance coefficient and the coding of quantization parameters will be discussed later using Figures 11 and 12.

[0041] In the case of Figure 9(b), that is, when the quantization control threshold is 512 pixels, there are no subblocks with more than 512 pixels, so the result is the same as in Figure 9(a).

[0042] In the case of Figure 9(c), that is, when the quantization control threshold is 256 pixels, the quantization parameters are shared within the right-hand block of Figure 9(c). Furthermore, within that block, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order. In this case, four quantization parameters are encoded.

[0043] In the case of Figure 9(d), that is, when the quantization control threshold is 128 pixels, the quantization parameters are shared within the right-hand block of the figure, and one quantization parameter is encoded in association with the subblock containing the significance coefficient first in the encoding order within that unit. Here, we will explain the encoding of quantization parameters in terminally partitioned subblocks. In Figure 9(d), in the terminally partitioned subblock located in the lower left, the number of pixels differs between the subblocks at both ends and the central subblock. Specifically, the two subblocks that are divided into quarters of the number of pixels have 64 pixels each, while the central subblock, which is divided into half the number of pixels, has 128 pixels. On the other hand, the value of the quantization control threshold is 128 pixels. In this case, since the number of pixels in the subblock with the largest number of pixels among the terminally partitioned subblocks is greater than or equal to the quantization control threshold, the quantization parameters are encoded in each of the three subblocks. Thus, in Figure 9(d), there are 7 quantization parameters to be encoded.

[0044] In the case of Figure 9(e), i.e., when the quantization control threshold is 64 pixels, the quantization parameters are shared within the right-hand block of the figure, and the quantization parameters are encoded in association with the subblock containing the significance coefficient first in the encoding order within that unit. In this case, 10 quantization parameters are encoded.

[0045] Here, the encoding of quantization parameters when using an asymmetric partition is explained using Figure 10. The thick lines and the rectangles within the thick lines in Figure 10 are the same as in Figure 9, so their explanation is omitted. The numbers in Figure 10 indicate the number of pixels in each subblock. Figures 10(a), (b), (c), and (d) show the cases where the quantization control threshold is 512 pixels, 256 pixels, 192 pixels, and 128 pixels, respectively. For clarity, in Figure 10, the quad partition in Figure 8(b) is used for the first partition of the recursive partition, dividing the basic block into four 16x16 pixel blocks, and an asymmetric partition is used for subsequent partitions. The four subblocks generated using the quad partition in Figure 8(b) are referred to here as the upper left region, upper right region, lower left region, and lower right region, based on their position in the basic block. The upper left region generates four subblocks in one partition using the quad tree partition in Figure 8(i). In the upper right and lower left regions, two subblocks are generated by asymmetric binary tree partitioning. Then, in the lower right region, three subblocks are generated by performing asymmetric binary tree partitioning twice.

[0046] In Figure 10(a), the quantization control threshold is 512 pixels, and the number of pixels in all subblocks in the figure is less than the quantization control threshold. Therefore, a common quantization parameter is used in all subblocks in the figure, and the quantization parameter is encoded in association with the subblock containing the first significance coefficient in the encoding order. In this case, only one quantization parameter is encoded.

[0047] In Figure 10(b), the quantization control threshold is 256 pixels. In the left side of Figure 10(b), the basic block is divided once using an asymmetric quadtree partition for the initial division, generating four subblocks. In this case, the four subblocks have 256 (16 × 16) pixels, which is greater than or equal to the quantization control threshold. Therefore, one quantization parameter is encoded in the upper left, upper right, lower left, and lower right regions. Subsequently, the upper left region is further divided once using an asymmetric quadtree partition, but since the number of pixels in each subblock is less than the quantization control threshold, the four subblocks share the quantization parameter. The upper right and lower left regions are divided into two subblocks of 192 pixels and 64 pixels using an asymmetric binary tree partition. However, since the number of pixels in each subblock is less than the quantization control threshold, the two subblocks in the upper right and lower left regions share the quantization parameter. In the lower right region, a 16x16 pixel subblock is first divided horizontally in a 1:3 ratio, generating a 64-pixel subblock and a 192-pixel subblock. Next, the 192-pixel subblock is further divided vertically in a 3:1 ratio, generating a 144-pixel subblock and a 48-pixel subblock. Meanwhile, the quantization control threshold is 256 pixels. In this case, of the two subblocks generated in the first binary tree division, the larger subblock on the right has 192 pixels, which is below the quantization control threshold. Furthermore, of the two subblocks generated in the second binary tree division, the larger subblock on the top has 144 pixels, which is also below the quantization control threshold. Therefore, the three subblocks share the quantization parameter. In this case, the quantization parameter is shared within the unit of the right-hand block in Figure 10(b). Furthermore, within that unit, one quantization parameter is encoded in association with the subblock containing the significance coefficient at the beginning of the encoding order. Similar to the example shown in Figure 9(c), in this case there are four quantization parameters to be encoded.

[0048] In Figure 10(c), the quantization control threshold is 192 pixels. Here, we will explain the encoding of quantization parameters when the subblock is divided into four parts using an asymmetric partition. On the left side of Figure 10(c), the upper left region is divided into four subblocks by a single asymmetric quadtree partition, each with a different number of pixels. Of the four subblocks, the one located in the lower left has 144 pixels and is the subblock with the most pixels. The subblock located in the upper right has 16 pixels and is the subblock with the fewest pixels. The subblocks located in the upper left and lower right each have 48 pixels. In contrast, the quantization control threshold is 192 pixels. Therefore, since the number of pixels in the subblock with the most pixels, 144 pixels, is less than the quantization control threshold, the four subblocks share the quantization parameters.

[0049] Here, we will further explain the encoding of quantization parameters when a subblock is divided into two using an asymmetric partition. In the case of an asymmetric binary tree partition, the subblock with the largest number of pixels among the two subblocks generated by performing the partition once is compared with the quantization control threshold. If the number of pixels in the subblock with the largest number of pixels is greater than or equal to the quantization control threshold, the quantization parameters are encoded in each subblock generated by the asymmetric partition. If the number of pixels in the subblock with the largest number of pixels is less than the quantization control threshold, the two subblocks generated by the asymmetric partition share the quantization parameters. For example, in the left side of Figure 10(c), the asymmetric binary tree partitioned subblocks located in the upper right and lower left have different numbers of pixels. The upper right region is a 16x16 pixel subblock that is binary tree partitioned vertically in a 3:1 ratio, and the lower left region is a binary tree partitioned horizontally in a 1:3 ratio. After partitioning, the larger subblock has 192 pixels, and the smaller subblock has 64 pixels. On the other hand, the quantization control threshold value is 192 pixels. In this case, of the subblocks generated by the asymmetric binary tree partitioning, the larger subblock has 192 pixels, which is greater than or equal to the quantization control threshold, so the quantization parameters are encoded in each of the two subblocks. Also, on the left side of Figure 10(c), the lower right region is divided into three subblocks by repeating the asymmetric binary tree partitioning twice. Specifically, a 16x16 pixel subblock is first binary-tree partitioned horizontally in a 1:3 ratio to generate a 64-pixel subblock and a 192-pixel subblock. Next, the 192-pixel subblock is further binary-tree partitioned vertically in a 3:1 ratio to generate a 144-pixel subblock and a 48-pixel subblock. On the other hand, the quantization control threshold value is 192 pixels. In this case, of the two subblocks generated in the first binary tree split, the larger subblock located on the right has 192 pixels, which is above the quantization control threshold, so the quantization parameters are encoded in each of the two subblocks. However, of the two subblocks generated in the second binary tree split, the larger subblock located on the top has 144 pixels, which is below the quantization control threshold.Therefore, in this case, as shown in the lower right block on the right side of Figure 10(c), two subblocks share one quantization parameter. Furthermore, within that block, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order. In this case, seven quantization parameters are encoded. Thus, if the number of pixels in a subblock compared to the quantization control threshold is less than the quantization control threshold, the target block for encoding the quantization parameters is changed.

[0050] In Figure 10(d), the quantization control threshold is 128 pixels. Here, we will explain the encoding of quantization parameters when the subblock is divided into four parts using an asymmetric partition. On the left side of Figure 10(d), the upper left region is divided into four subblocks by one asymmetric quadtree partition, each with a different number of pixels. Of the four subblocks, the one located in the lower left has 144 pixels and is the subblock with the most pixels. The subblock located in the upper right has 16 pixels and is the subblock with the fewest pixels. The subblocks located in the upper left and lower right each have 48 pixels. On the other hand, the value of the quantization control threshold is 128 pixels. In this case, the subblock with the most pixels has 128 pixels, which is greater than or equal to the quantization control threshold, so the quantization parameters are encoded in each of the four subblocks. Next, on the left side of Figure 10(d), the lower right region is divided into two asymmetric binary tree partitions. The first binary tree splitting involves first splitting a 16x16 pixel subblock horizontally in a 1:3 ratio, generating a 64-pixel subblock and a 192-pixel subblock. In the subsequent second binary tree splitting, the 192-pixel subblock is further split vertically in a 3:1 ratio, generating a 144-pixel subblock and a 48-pixel subblock. Meanwhile, the quantization control threshold is 128 pixels. Therefore, of the two subblocks generated in the second binary tree splitting, the larger upper subblock has 144 pixels, which is greater than or equal to the quantization control threshold. Thus, in the lower right region, the quantization parameters are encoded in each of the three subblocks. In this case, the quantization parameters are shared within the block units on the right side of Figure 10(d). Furthermore, within that block unit, one quantization parameter is encoded in association with the subblock containing the significance coefficient at the beginning of the encoding order. In this case, there are 11 quantization parameters encoded, as shown on the right side of Figure 10(d).

[0051] Here, we will explain the encoding of quantization parameters when performing multiple asymmetric binary tree partitions to create subblocks that have the same shape as a single ternary partition, using Figure 17. The thick lines and rectangles within the thick lines in Figure 17 are the same as in Figure 9, so we will omit their explanation. The numbers in Figure 17 indicate the number of pixels in each subblock. Figures 17(a) to (d) show the cases where the quantization control thresholds are 1024, 768, 768, and 512, respectively. In the case of Figure 17(a), first the basic block is partitioned horizontally in a 1:3 ratio binary tree. Of the two subblocks generated by the partition, the larger subblock on the right has 768 pixels, which is less than the quantization control threshold of 1024. Next, the larger subblock on the right obtained in the first partition is partitioned horizontally in a 2:1 ratio in the second binary tree partition. As a result, of the two subblocks generated in the second split, the larger subblock located on the left has 512 pixels, which is below the quantization control threshold. Therefore, as shown in the right-hand diagram of Figure 17(a), there is only one quantization parameter to be encoded. Alternatively, the first binary tree split can be performed by dividing the horizontal direction in a 3:1 ratio to generate the three subblocks shown in Figure 17(a). In this case, the larger subblock located on the left has 768 pixels, which is below the quantization control threshold of 1024. Next, the larger subblock located on the left obtained in the first split is divided horizontally in a 1:2 ratio in the second binary tree split. As a result, of the two subblocks generated in the second split, the larger subblock located on the right has 512 pixels, which is below the quantization control threshold. Therefore, in the case of Figure 17(a), regardless of the horizontal splitting ratio in the first binary tree split, there is only one quantization parameter to be encoded, as shown in the right-hand diagram of Figure 17(a).

[0052] In the case of Figure 17(b), the first asymmetric binary tree partition is performed to divide the tree horizontally in a 1:3 ratio. The larger subblock on the right side of the resulting subblock has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Next, the larger subblock on the right side generated in the first partition is divided horizontally in a 2:1 ratio using a second asymmetric binary tree partition. The larger subblock on the left side of the resulting subblock has 512 pixels, which is less than the quantization control threshold. Therefore, the subblock generated in the second partition is encoded with a common quantization parameter, as shown in the right-hand diagram of Figure 17(b). In other words, in this case, two quantization parameters are encoded for the basic block.

[0053] In the case of Figure 17(c), the first asymmetric binary tree partition is performed to divide the tree horizontally in a 3:1 ratio. The larger subblock on the left side of the resulting subblock has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Next, the larger subblock on the left side generated in the first partition is divided horizontally in a 1:2 ratio using a second asymmetric binary tree partition. The larger subblock on the right side of the resulting subblock has 512 pixels, which is less than the quantization control threshold. Therefore, the subblock generated in the second partition is encoded with a common quantization parameter, as shown in the right-hand diagram of Figure 17(c). In other words, in this case, two quantization parameters are encoded for the basic block.

[0054] In both Figure 17(b) and Figure 17(c), the quantization control threshold is 768, and two quantization parameters are encoded for the basic block. However, of the three subblocks, in Figure 17(b), the central and right subblocks share the quantization parameter, while in Figure 17(c), the left and central subblocks share the quantization parameter.

[0055] In the case of Figure 17(d), the first asymmetric binary tree partition is performed to divide the tree horizontally in a 1:3 ratio. The larger subblock on the right side generated by the partition has 768 pixels, which is above the quantization control threshold. Next, the larger subblock on the right side generated by the first asymmetric binary tree partition is divided in a 2:1 ratio by the second asymmetric binary tree partition. In this case, the larger subblock on the left side of the two subblocks generated by the binary tree partition has 512 pixels, which is above the quantization control threshold. Therefore, the three subblocks on the left side of Figure 17(d) encode quantization parameters in each subblock, as shown in the right side of Figure 17(d). That is, three quantization parameters are encoded for the basic block. It is also possible to perform the first asymmetric binary tree partition in a 3:1 ratio horizontally. Subsequently, the larger subblock on the left side generated by the first partition can also be divided in a 1:2 ratio by the second asymmetric binary tree partition. Even in this case, as shown in the right-hand diagram of Figure 17(d), three quantization parameters are encoded for the basic block.

[0056] Here, we will explain the encoding of quantization parameters when performing two asymmetric binary tree partitions so that the partition shape is the same as that of a single asymmetric quadtree partition, using Figure 18. The thick lines and the rectangles within the thick lines in Figure 18 are the same as in Figure 9, so we will omit their explanation. The numbers in Figure 18 indicate the number of pixels in each subblock. The arrows shown in the right-hand figures of Figures 18(a) to (d) indicate the encoding order. Figures 18(a) and (b) show the case where the quantization control threshold is 768, and Figures 18(c) and (d) show the case where the quantization control threshold is 512. Also, in Figures 18(a) and (c), the first partition is a vertically asymmetric binary tree partition, and the subsequent second partition is a horizontally asymmetric binary tree partition. In contrast, Figures 18(b) and 18(d) show that the first partition is a horizontally asymmetric binary tree partition, and the subsequent second partition is a vertically asymmetric binary tree partition.

[0057] In the case of Figure 18(a), first, a flag of 1 is encoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 1 is encoded to indicate that it is a vertical partition, and a flag of 1 is encoded to indicate that the partition ratio is 1:3. Then, as the first partition, an asymmetric binary tree partition is performed to divide the base block into two subblocks in a vertical ratio of 1:3. In this case, the larger subblock located at the bottom has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Therefore, the smaller subblock located at the top and the larger subblock located at the bottom do not share quantization parameters. Subsequently, for the smaller subblock located at the top generated in the first partition, a flag of 0 is encoded to indicate that it is a horizontal partition, and a flag of 0 is encoded to indicate that the partition ratio is 3:1. Then, as the second partition, an asymmetric binary tree partition is performed to divide the smaller subblock located at the top generated in the first partition into two subblocks in a horizontal ratio of 3:1. In this case, the larger subblock on the left, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 768. Therefore, the 192-pixel subblock on the left and the 64-pixel subblock on the right share the quantization parameters and encode one quantization parameter. Additionally, for the larger subblock on the lower side, generated in the first split, a flag of 0 is encoded to indicate that it is a horizontal split, and a flag of 0 is encoded to indicate that the split ratio is 3:1. For the second split, an asymmetric binary tree split is performed, dividing the larger subblock on the lower side, generated in the first split, into two subblocks horizontally in a 3:1 ratio. In this case, the larger subblock on the left, generated in the second split, has 576 pixels, which is less than the quantization control threshold of 768. Therefore, the 576-pixel subblock on the left and the 192-pixel subblock on the right share the quantization parameters and encode one quantization parameter. Therefore, in this case, two quantization parameters are encoded for the basic block, as shown in the central figure of Figure 18(a).Furthermore, the four subblocks generated by the partition are encoded in the order shown in the right-hand diagram of Figure 18(a). At this time, for each block unit that shares a quantization parameter, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order.

[0058] In the case of Figure 18(b), first, a flag of 1 is encoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 0 is encoded to indicate that it is a horizontal partition, and a flag of 0 is encoded to indicate that the partition ratio is 3:1. Then, as the first partition, an asymmetric binary tree partition is performed to divide the base block into two subblocks horizontally in a ratio of 3:1. In this case, the larger subblock on the left has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Therefore, the larger subblock on the left and the smaller subblock on the right do not share quantization parameters. Subsequently, for the larger subblock on the left generated in the first partition, a flag of 1 is encoded to indicate that it is a vertical partition, and a flag of 1 is encoded to indicate that the partition ratio is 1:3. Then, as the second partition, an asymmetric binary tree partition is performed to divide the larger subblock on the left generated in the first partition vertically into two subblocks in a ratio of 1:3. In this case, the larger subblock located on the lower side, generated in the second split, has 576 pixels, which is less than the quantization control threshold of 768. Therefore, the 192-pixel subblock located on the upper side and the 576-pixel subblock located on the lower side share the quantization parameters and encode one quantization parameter. Additionally, for the smaller subblock located on the right side, generated in the first split, a flag of 1 is encoded to indicate that it is a vertical split, and a flag of 1 is encoded to indicate that the split ratio is 1:3. Then, as the second split, an asymmetric binary tree split is performed, dividing the smaller subblock located on the right side, generated in the first split, into two subblocks vertically in a 1:3 ratio. In this case, the larger subblock located on the lower side, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 768. Therefore, the 64-pixel subblock located on the upper side and the 192-pixel subblock located on the lower side share the quantization parameters and encode one quantization parameter. Therefore, in this case, two quantization parameters are encoded for the basic block, as shown in the central diagram of Figure 18(b).Furthermore, the four subblocks generated by the partition are encoded in the order shown in the right-hand diagram of Figure 18(b). At this time, for each block unit that shares a quantization parameter, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order.

[0059] In the case of Figure 18(c), first, a flag of 1 is encoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 1 is encoded to indicate that it is a vertical partition, and a flag of 1 is encoded to indicate that the partition ratio is 1:3. Then, as the first partition, an asymmetric binary tree partition is performed to divide the base block into two subblocks in a vertical ratio of 1:3. In this case, the larger subblock located at the bottom has 768 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the smaller subblock located at the top and the larger subblock located at the bottom do not share quantization parameters. Subsequently, for the smaller subblock located at the top generated in the first partition, a flag of 0 is encoded to indicate that it is a horizontal partition, and a flag of 0 is encoded to indicate that the partition ratio is 3:1. Then, as the second partition, an asymmetric binary tree partition is performed to divide the smaller subblock located at the top generated in the first partition into two subblocks in a horizontal ratio of 3:1. In this case, the larger subblock on the left, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 512. Therefore, the 192-pixel subblock on the left and the 64-pixel subblock on the right share the quantization parameters and encode one quantization parameter. Additionally, for the larger subblock on the lower side, generated in the first split, a flag of 0 is encoded to indicate that it is a horizontal split, and a flag of 0 is encoded to indicate that the split ratio is 3:1. For the second split, an asymmetric binary tree split is performed, dividing the larger subblock on the lower side, generated in the first split, into two subblocks horizontally in a 3:1 ratio. In this case, the larger subblock on the left, generated in the second split, has 576 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the quantization parameters are encoded for each of the 576-pixel subblock on the left and the 192-pixel subblock on the right. Thus, in this case, three quantization parameters are encoded for the basic block, as shown in the central figure of Figure 18(c).Furthermore, the four subblocks generated by the partition are encoded in the order shown in the right-hand diagram of Figure 18(c). At this time, for each block unit that shares a quantization parameter, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order.

[0060] In the case of Figure 18(d), first, a flag of 1 is encoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 0 is encoded to indicate that it is a horizontal partition, and a flag of 0 is encoded to indicate that the partition ratio is 3:1. Then, as the first partition, an asymmetric binary tree partition is performed to divide the base block into two subblocks horizontally in a ratio of 3:1. In this case, the larger subblock on the left has 768 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the larger subblock on the left and the smaller subblock on the right do not share quantization parameters. Subsequently, for the larger subblock on the left generated in the first partition, a flag of 1 is encoded to indicate that it is a vertical partition, and a flag of 1 is encoded to indicate that the partition ratio is 1:3. Then, as the second partition, an asymmetric binary tree partition is performed to divide the larger subblock on the left generated in the first partition vertically into two subblocks in a ratio of 1:3. In this case, the larger subblock located on the lower side, generated in the second split, has 576 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the quantization parameters are encoded in both the 192-pixel subblock located on the upper side and the 576-pixel subblock located on the lower side. Additionally, for the smaller subblock located on the right side, generated in the first split, a flag of 1 is encoded to indicate that it is a vertical split, and a flag of 1 is encoded to indicate that the split ratio is 1:3. For the second split, an asymmetric binary tree split is performed, dividing the smaller subblock located on the right side, generated in the first split, into two subblocks vertically in a 1:3 ratio. In this case, the larger subblock located on the lower side, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 512. Therefore, the 64-pixel subblock located on the upper side and the 192-pixel subblock located on the lower side share the quantization parameters, and a single quantization parameter is encoded. Therefore, in this case, three quantization parameters are encoded for the basic block, as shown in the central figure of Figure 18(d).Furthermore, the four subblocks generated by the partition are encoded in the order shown in the right-hand diagram of Figure 18(d). At this time, for each block unit that shares a quantization parameter, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order.

[0061] In this embodiment, we have described a case where two asymmetric binary tree partitions are performed to create subblock partitions that result in the same partition shape as a single asymmetric quad tree partition. In this case, for both the first and second asymmetric binary tree partitions, a flag indicating the execution of a vertical or horizontal partition and a flag for specifying the partition ratio are encoded. However, this embodiment is not limited to this. It is not necessary to encode a flag indicating the execution of a vertical or horizontal partition in the second asymmetric binary tree partition. For example, in the partitions of Figures 18(a) and 18(c), the flag representing the execution of a vertical partition is encoded as 1 in the first asymmetric binary tree partition, and the flag indicating the execution of a vertical or horizontal partition is not encoded in the second asymmetric binary tree partition. The value of the flag indicating the execution of a vertical or horizontal partition used in the first asymmetric binary tree partition, which is 1, may be referenced, and a horizontal partition, which is the inverse value of 0, may be executed. Alternatively, in the partitions shown in Figures 18(a) and 18(c), the flag 1 is encoded as a flag indicating the execution of a vertical partition in the first asymmetric binary tree partition. Of the two subblocks obtained in the first asymmetric binary tree partition, the upper subblock may also encode a flag 1 indicating the execution of a vertical partition. Furthermore, the lower subblock may not encode a flag indicating the execution of a vertical or horizontal partition, but instead refer to the value of the flag 1 encoded in the upper subblock, which indicates the execution of a vertical partition, and execute a vertical partition. Alternatively, a horizontal partition, which is the opposite value 0, may be executed. Note that if both the upper and lower subblocks obtained in the first asymmetric binary tree partition that partitions vertically are partitioned vertically, the base block will be divided into four subblocks vertically as a result. Alternatively, in the partitions shown in Figures 18(b) and 18(d), the first asymmetric binary tree partition encodes 0 as a flag indicating the execution of a horizontal partition, while the second asymmetric binary tree partition does not encode a flag indicating the execution of a vertical or horizontal partition. The value of the flag indicating the execution of a vertical or horizontal partition used in the first asymmetric binary tree partition, which is 0, may be referenced, and a vertical partition, which is the inverse value of 1, may be executed.Alternatively, in the partitions shown in Figures 18(b) and 18(d), the flag 0 is encoded as a flag indicating the execution of a horizontal partition in the first asymmetric binary tree partition. Of the two subblocks obtained in the first asymmetric binary tree partition, the left-hand subblock may also encode a flag 0 indicating the execution of a horizontal partition. Furthermore, the right-hand subblock may not encode a flag indicating the execution of a vertical or horizontal partition, but instead refer to the value of the flag 0 encoded in the left-hand subblock indicating the execution of a horizontal partition, and perform a horizontal partition. Or, a vertical partition, which is the opposite value 1, may be performed. Note that if both the left-hand and right-hand subblocks obtained in the first asymmetric binary tree partition that partitions horizontally are horizontally partitioned, the basic block will be divided into four subblocks horizontally as a result. Also, in the second asymmetric binary tree partition, it is not necessary to encode a flag to specify the partition ratio. For example, in the partitions shown in Figures 18(a) and 18(c), in the upper subblock of the two subblocks obtained from the first asymmetric binary tree partition, a flag of 0 is encoded to indicate that the partition ratio is 3:1. In the lower subblock, it is not necessary to encode a flag to specify the partition ratio. In this case, the partition ratio of the upper subblock may be used as the partition ratio of the lower subblock. Alternatively, in the partitions shown in Figures 18(b) and 18(d), in the left subblock of the two subblocks obtained from the first asymmetric binary tree partition, a flag of 1 is encoded to indicate that the partition ratio is 1:3. In the right subblock, it is not necessary to encode a flag to specify the partition ratio. In this case, the partition ratio of the left subblock may be used as the partition ratio of the right subblock. Thus, flags indicating the execution of vertical or horizontal partitioning required for the second asymmetric binary tree partition, or flags to specify the partition ratio, are not encoded.This allows the flags indicating whether a vertical or horizontal partition is performed, and the flags for specifying the partition ratio, to be represented with less coding than when they are encoded in the first and second asymmetric binary tree partitions.

[0062] In this embodiment, a flag indicating the use of an asymmetric binary tree partition is encoded in the first block partition of the basic block, but this embodiment is not limited to this. The flag indicating the use of an asymmetric binary tree partition may be encoded in the picture data portion before encoding each basic block. For example, one 1 may be encoded in the sequence header portion or picture header portion as a flag indicating the use of an asymmetric binary tree partition. Then, the value encoded in the sequence header portion or picture header portion may be used in each basic block without encoding the flag indicating the use of an asymmetric binary tree partition. In this way, by not encoding the flag indicating the use of an asymmetric binary tree partition in each basic block, but encoding one in the sequence header portion or picture header portion, subblock partitioning due to asymmetric partitioning can be represented with less coding.

[0063] In the above, it was stated that a flag indicating the use of an asymmetric binary tree partition is encoded first, followed by a flag indicating the execution of a vertical or horizontal partition. However, this embodiment is not limited to this. A flag indicating the use of an asymmetric binary tree partition may be encoded after encoding a flag indicating the execution of a vertical or horizontal partition. For example, consider the case where a basic block is partitioned once vertically using an asymmetric binary tree partition. In the second partition, 1 may be encoded as a flag indicating the execution of a vertical partition, followed by 1 as a flag indicating the use of an asymmetric binary tree partition. Alternatively, in the second partition of the two subblocks generated in the first partition, 0 may be encoded as a flag indicating the execution of a horizontal partition, followed by 0 as a flag indicating the use of a symmetric binary tree partition. In this way, in each subblock, a flag indicating the execution of a vertical or horizontal partition is encoded first, followed by a flag indicating the use of an asymmetric binary tree partition. This allows for selection of whether to use an asymmetric or symmetric partition in the partition of each subblock, enabling finer control over the subblock partitioning.

[0064] Furthermore, Figure 19 illustrates the encoding of quantization parameters when subblock partitioning is performed multiple times using ternary tree partitioning or asymmetric binary tree partitioning to obtain the same partition shape as a single quadary tree partition. The thick lines and rectangles within the thick lines in Figure 19 are the same as in Figure 9, so no explanation is provided. The numbers in Figure 19 indicate the number of pixels in each subblock. Figures 19(a) to (d) show the cases where the quantization control thresholds are 1024, 640, 640, and 256, respectively.

[0065] In the case of Figure 19(a), the first partition is a ternary tree partition, dividing the data vertically into three subblocks in a ratio of 1:6:1. The central subblock, which has the largest number of pixels, has 768 pixels, which is less than the quantization control threshold of 1024. Next, as the second partition, an asymmetric binary tree partition is performed, dividing the central subblock vertically in a ratio of 2:1. In this case, the larger upper subblock generated by the second partition has 512 pixels, which is less than the quantization control threshold of 1024. Therefore, in this case, as shown in the right-hand diagram of Figure 19(a), one quantization parameter common to the four subblocks is encoded. Alternatively, as the first partition, an asymmetric binary tree partition can be performed, dividing the data vertically in a ratio of 1:7. Then, in the second partition, the larger lower subblock generated by the first partition can be binary tree partitioned vertically in a ratio of 4:3. Furthermore, in the third partition, the smaller subblock located at the bottom, generated in the second partition, can be vertically partitioned into a binary tree in a 2:1 ratio. Even in this case, one quantization parameter is encoded, as shown in the right-hand diagram of Figure 19(a).

[0066] In the case of Figure 19(b), an asymmetric binary tree partitioning is performed as shown in Figure 7(j), with a vertical division in a 5:3 ratio. In this case, the larger subblock located on the upper side has 640 pixels, which is greater than or equal to the quantization control threshold of 640. Therefore, the two subblocks generated in the first partitioning do not share quantization parameters. Next, the larger subblock located on the upper side generated in the first partitioning is further divided vertically in a 1:4 ratio binary tree partitioning. In this case, the larger subblock located on the lower side has 512 pixels, which is less than the quantization control threshold. Also, the smaller subblock located on the lower side of the subblock generated in the first partitioning is further divided vertically in a 2:1 ratio binary tree partitioning. In this case, the larger subblock located on the upper side has 256 pixels, which is less than the quantization control threshold. Therefore, in this case, two quantization parameters are encoded, as shown in the right-hand diagram of Figure 19(b).

[0067] In the case of Figure 19(c), the first partition is a ternary tree partition, dividing the data into three subblocks in a vertical ratio of 1:6:1. In this case, the central subblock, which has the largest number of pixels, has 768 pixels, which is greater than or equal to the quantization control threshold of 640. Next, an asymmetric binary tree partition is performed, dividing the central subblock from the subblocks generated in the first partition in a vertical ratio of 2:1. In this case, the larger subblock located at the top has 512 pixels, which is less than the quantization control threshold. Therefore, the two subblocks generated in the second partition share the quantization parameters. Consequently, in this case, three quantization parameters are encoded for the basic block, as shown in the right-hand diagram of Figure 19(c).

[0068] As shown in Figures 19(b) and 19(c), even when the basic block is similarly divided into four subblocks and the same quantization control threshold is used, it becomes possible to encode a different number of quantization parameters for the basic block. This is due to the difference in the combination of ternary tree partitioning and asymmetric binary tree partitioning. As a result, quantization parameters can be controlled more finely.

[0069] In the case of Figure 19(d), the first partition is a ternary tree partition, dividing the data into three subblocks in a vertical ratio of 1:6:1. The central subblock, which has the largest number of pixels, has 768 pixels, which is greater than or equal to the quantization control threshold of 256. Next, as the second partition, an asymmetric binary tree partition is performed, dividing the central subblock vertically in a ratio of 2:1. In this case, the larger upper subblock generated by the second partition has 512 pixels, which is greater than or equal to the quantization control threshold of 256. In this case, as shown in the right-hand diagram of Figure 19(d), quantization parameters are encoded in each subblock. Therefore, four quantization parameters are encoded for the basic block. Alternatively, as the first partition, an asymmetric binary tree partition can be performed, dividing the data vertically in a ratio of 7:1. Then, in the second partition, the larger upper subblock generated by the first partition can be binary-tree partitioned vertically in a ratio of 1:6. Furthermore, in the third partition, the larger subblock located at the bottom, generated in the second partition, can be vertically partitioned into a binary tree in a 2:1 ratio. Even in this case, four quantization parameters are encoded, as shown in the right-hand diagram of Figure 19(d).

[0070] In the above description, the quantization control threshold is compared with the subblock having the maximum number of pixels obtained in a single subblock division, but this embodiment is not limited to this. The quantization parameters may be encoded based on the result of comparing the quantization control threshold with the subblock having the minimum number of pixels obtained in a single subblock division.

[0071] The case of comparing with the subblock with the largest number of pixels is explained using Figure 20(a). In this case, the quantization control threshold is 192 pixels. The thick lines and the rectangles within the thick lines in Figure 20 are the same as in Figure 9, so no explanation is given. The numbers in Figure 20 indicate the number of pixels in each subblock. For clarity, in Figure 20, as in Figure 10, the quad partition of Figure 8(b) is used for the first partition of the recursive partition, dividing the basic block into four 16x16 pixel blocks. Also, based on the position in the basic block, these are referred to here as the upper left region, upper right region, lower left region, and lower right region. In Figures 20(a) and (b), the upper left region on the left side is partitioned vertically with an asymmetric binary tree partition in a ratio of 5:3. The upper right region in the same figure is partitioned horizontally with an asymmetric binary tree partition in a ratio of 1:3. The lower left region in the same figure is partitioned horizontally using a ternary partition in a ratio of 1:2:1. The area in the lower right of the figure is partitioned using an asymmetric quadtree partition, as shown in Figure 8(i), which generates four subblocks in a single partition.

[0072] In the upper left, lower left, and lower right regions of the left side of Figure 20(a), the subblocks with the largest number of pixels are 160, 128, and 144 pixels, respectively, and the number of pixels in each subblock is less than the quantization control threshold of 192. Therefore, one quantization parameter is encoded in the upper left, lower left, and lower right regions. On the other hand, in the upper right region, the subblock with the largest number of pixels is 192 pixels, which is above the quantization control threshold. Therefore, two quantization parameters are encoded in the upper right region, as shown in the right side of Figure 20(a). In other words, five quantization parameters are encoded in the entire right side of Figure 20(a).

[0073] The case of comparing with the subblock having the minimum number of pixels will be explained using Figure 20(b). In this case, the quantization control threshold is 96 pixels. The division state of the basic block in Figure 20(b) is the same as in Figure 20(a), so the explanation will be omitted. In the upper right, lower left, and lower right regions on the left side of Figure 20(b), the subblocks with the minimum number of pixels are 64 pixels, 64 pixels, and 16 pixels, respectively, and the number of pixels in all of these subblocks is less than the quantization control threshold of 96. Therefore, in the upper right, lower left, and lower right regions, one quantization parameter is encoded. On the other hand, in the upper left region, the subblock with the minimum number of pixels is 96, which is above the quantization control threshold. Therefore, in the upper left region, two quantization parameters are encoded, as shown in the figure on the right side of Figure 20(b). As a result, in the case of Figure 20(b), five quantization parameters are encoded. In this way, it becomes possible to share quantization parameters between different combinations of subblocks within a basic block divided into multiple subblocks. As a result, the quantization parameters can be controlled more precisely.

[0074] Furthermore, although this embodiment describes the comparison as being made with the number of pixels, it may also be made with an index that substitutes for the number of pixels in a subblock. Specifically, when the number of pixels in a subblock is a power of 2 and the exponent of 2 is an integer, the quantization control threshold may be compared with the exponent. The comparison using the quantization control threshold will be explained using the table shown in Figure 21. However, the values ​​shown in Figure 21 are just examples, and this embodiment is not limited thereto. The first column of Figure 21 is the number of pixels in a subblock generated by the division shape shown in Figure 8 or a recursive combination thereof. The second column is the index corresponding to the exponent when the number of pixels in a subblock is expressed as a power of 2. There is a relationship between the number of pixels in a subblock and the index corresponding to the exponent, where the index decreases by 1 when the number of pixels in a subblock is halved. For example, the index corresponding to a number of pixels of 512 is 9, and the index corresponding to a number of pixels of 256 is 8. Furthermore, when the left side of Figure 9(e) is represented using exponents of 2, the subblock pixel counts of 64, 128, and 256 correspond to indices 6, 7, and 8, respectively, and the quantization control threshold is 6. In this case, the quantization parameters are shared by the block units on the right side of the figure, and the quantization parameters are encoded in association with the subblock containing the significance coefficient first in the coding order for that unit. Thus, even when comparing the quantization control threshold with an index that substitutes for the number of pixels in a subblock, the quantization parameters can be controlled in the same way as when comparing the quantization control threshold with the number of pixels. Since the value representing the quantization control threshold is smaller than the index that substitutes for the number of pixels, the quantization control threshold can be represented with less coding, especially when coding using Golomb coding.

[0075] Returning to Figure 1, the inverse quantization / inverse transform unit 106 inversely quantizes the input residual coefficients to reconstruct the transformation coefficients, and then performs an inverse orthogonal transform on the reconstructed transformation coefficients to reconstruct the prediction error, which is then output to the image reconstruction unit 107. The same quantization parameters used in the transformation / quantization unit 105 are used in the inverse quantization processing of each subblock.

[0076] The image playback unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104 and plays back the predicted image. Then, it plays back the image data from the played back predicted image and the played back prediction error input from the inverse quantization / inverse transform unit 106, inputs it into the frame memory 108, and stores it.

[0077] The in-loop filter unit 109 reads the regenerated image from the frame memory 108 and performs in-loop filtering, such as deblocking filtering. In-loop filtering is performed based on the prediction mode of the prediction unit 104, the values ​​of the quantization parameters used in the transformation / quantization unit 105, the presence or absence of non-zero values ​​(hereinafter referred to as significance coefficients) in the processing subblocks after quantization, or subblock division information. The filtered image is then input back into the frame memory 108 and re-stored.

[0078] The encoding unit 110 entropy encodes the residual coefficients generated by the transformation / quantization unit 105 and the prediction information input from the prediction unit 104 in block units to generate coded data.

[0079] Entropy coding methods include Golomb coding, arithmetic coding, and Huffman coding, and no particular method is specified. The generated coded data is output to the integrated coding unit 111. In coding the quantization parameters that constitute the quantization information, an identifier indicating the difference between the quantization parameter of the subblock to be coded and the predicted value calculated using the quantization parameters of subblocks coded before that subblock is coded is coded. In this embodiment, the quantization parameter coded immediately before the subblock in coding order is used as the predicted value, and the difference between that and the quantization parameter of the subblock is calculated, but the predicted value of the quantization parameter is not limited to this. The quantization parameter of the subblock adjacent to the left or above the subblock may be used as the predicted value, or a value calculated from the quantization parameters of multiple subblocks, such as the average value, may be used as the predicted value. Furthermore, if the subblock to be processed is the first subblock in coding order among the subblocks belonging to the first basic block in a basic block row, the quantization parameter of the subblock in the basic block directly above it may be used as the predicted value. This enables parallel processing on a basic block row basis. Note that the first basic block in a basic block row refers to the basic block that has a picture boundary or tile boundary on its left side.

[0080] Here, the process of encoding quantization parameters into each subblock based on the quantization control threshold will be further explained using Figures 11 and 12. The left-hand diagrams in Figures 11(a) to (f) and 12(a) to (d) show the types of block partitioning and the quantization parameters (Qp) used in each subblock. The left-hand diagram in Figure 12(a) shows the four subblocks generated by partitioning the basic block once using the asymmetric quadtree partitioning shown in Figure 7(i). In contrast, the left-hand diagram in Figure 12(b) shows the basic block first partitioned once using a binary tree partitioning that divides horizontally in a 3:1 ratio to generate two subblocks. Furthermore, it shows the four subblocks generated by partitioning the two subblocks obtained in the first partition using a binary tree partitioning that divides vertically in a 1:3 ratio. Subblocks with diagonal lines indicate subblocks associated with the quantization parameters to be encoded. The thick-lined frame indicates the region where the quantization parameters determined based on the quantization control threshold and the number of pixels in the subblock to be processed are shared.

[0081] The central figures in Figures 11(a)-(f) and 12(a)-(d) indicate whether each subblock has a significance coefficient. As mentioned above, a significance coefficient is a non-zero coefficient among the residual coefficients after transformation and quantization. That is, "has a significance coefficient" means that there is at least one non-zero residual coefficient in the subblock after transformation and quantization. Also, the arrows shown in the right-hand figures in Figures 11(a)-(f) and 12(a)-(d) indicate the coding order. In this embodiment, within the region where quantization parameters are shared, the quantization parameters are coded in association with the subblock that first contains the significance coefficient in the coding order. For example, in Figures 11(b) and 12(a), the first subblock to contain the significance coefficient in the coding order is the upper right subblock, so the quantization parameters are coded in association with that subblock. In this case, in the lower left and lower right subblocks, the quantization parameters are not coded in those quantization parameter coding units because the quantization parameters have already been coded in the upper right subblock. On the other hand, in the quantization and inverse quantization processes in the lower left and lower right subblocks, the same quantization parameter QpA as in the upper right subblock is used. Furthermore, since there is no significance coefficient in the upper left subblock, inverse quantization is not performed, but processes that use quantization parameters, such as deblocking filters, use the same quantization parameter QpA as in the upper right subblock.

[0082] Furthermore, in Figure 11(f), the subblock containing the significance coefficient first in the coding order is the lower subblock; therefore, the quantization parameter is associated with and coded for this subblock, while the quantization parameter is not coded for the upper and middle subblocks. However, in the upper and middle subblocks of Figure 11(f), similar to the upper left subblock of Figure 11(b), the same quantization parameter QpA as the lower subblock is used for processes that utilize quantization parameters, such as deblocking filters. Also, in Figure 12(b), similar to Figure 12(a), the upper left subblock does not contain a significance coefficient, but the coding order is different because the generation process of the four subblocks is different. In Figure 12(b), the subblock containing the significance coefficient first in the coding order is the lower left subblock, and the quantization parameter is associated with and coded for this subblock. Since the upper left subblock of Figure 12(b) does not contain a significance coefficient, the quantization parameter is not coded for it. On the other hand, in Figure 12(b), significance coefficients exist in the upper right and lower right subblocks, but because the quantization parameters are encoded in the lower left subblock, the quantization parameters are not encoded in the upper right and lower right subblocks of Figure 12(b). Furthermore, in the quantization and dequantization processes in the upper right and lower right subblocks, the same quantization parameter QpA as in the lower left subblock is used. On the other hand, since there is no significance coefficient in the upper left subblock, the dequantization process is not performed, similar to Figure 12(a), but the same quantization parameter QpA as in the lower left subblock is used for processes that use quantization parameters, such as deblocking filters. Also, in Figure 12(d), since there is no significance coefficient in the upper subblock, the quantization parameters are not encoded. On the other hand, since there is a significance coefficient in the lower subblock of Figure 12(d), the quantization parameters are encoded. In the upper subblock of Figure 12(d), similar to Figure 12(b), the same quantization parameter QpA as in the lower subblock is used for processes that use quantization parameters, such as deblocking filters.

[0083] Thus, in the subblocks within the region where the quantization parameters determined by the quantization control threshold are shared, the quantization parameters are associated with and encoded in the subblock containing the significance coefficient first in the encoding order.

[0084] Furthermore, Figure 13 illustrates the case where there are subblocks in the region where quantization parameters are shared that do not contain any significance coefficients. The left side of Figure 13(a) shows a subblock partitioning similar to that in Figure 9, and the numbers in the figure indicate the number of pixels in each subblock. The left side of Figure 13(b) shows a subblock partitioning similar to that in Figure 10, and the numbers in the figure indicate the number of pixels in each subblock. The right side of Figure 13 is an example showing a shared region of quantization parameters determined based on the quantization control threshold and the number of pixels in each subblock, similar to Figures 9 and 10. Note that Figure 13(a) is an example where the quantization control threshold is 256, and Figure 13(b) is an example where it is 192. In Figure 13(a), we will explain the case where there are no significance coefficients in any of the three subblocks in the region corresponding to the quantization parameter QpD. In this case, the quantization parameter QpD for these three subblocks is not encoded. However, for processes such as deblocking filters, the same value as the quantization parameter encoded immediately before, i.e., QpC, is used. On the other hand, in Figure 13(b), it is assumed that there are no significance coefficients for both of the two subblocks in the region corresponding to the quantization parameter QpG. In this case, the quantization parameter QpG is not encoded in those two subblocks. For processes that use quantization parameters, such as deblocking filters, the QpF encoded immediately before is used. Although it is stated that in processes that use quantization parameters in a quantization parameter encoding unit where no significance coefficient exists, the quantization parameter encoded immediately before in the encoding order is used, this embodiment is not limited to this. For example, in Figure 13(a), QpB, which is the quantization parameter of the quantization parameter encoding unit adjacent to the top, may be used, or QpC, which is the quantization parameter adjacent to the left, may be used. In Figure 13(b), the quantization parameter of the quantization parameter encoding unit adjacent to the top is QpC, and the quantization parameter adjacent to the left is QpF. Alternatively, a value calculated from the quantization parameters of multiple quantization parameter encoding units, such as the mean value, may be used. Furthermore, initial values ​​for quantization parameters can be used for tiles or tile groups composed of multiple tiles.A tile is a unit that divides a frame, and it consists of at least one basic block.

[0085] Next, using Figure 14, we will explain how to encode quantization parameters in a subblock that has been partitioned into a ternary tree, and then using Figure 16, we will explain how to encode quantization parameters in a subblock that has been partitioned into an asymmetrical quadary tree. The meaning of the subblocks with diagonal lines and the frames with thick lines in Figures 14 and 16 is the same as in Figure 11, so we will omit the explanation.

[0086] In Figure 14(a), all three subblocks contain significance coefficients. In this case, the quantization parameters are encoded in each of the three subblocks. That is, QpA is encoded in the first subblock, QpB in the second subblock, and QpC in the third subblock. In Figure 16(a), all four subblocks contain significance coefficients, and the quantization parameters are encoded in each of the subblocks. That is, QpA, QpB, QpC, and QpD are encoded in the four subblocks in the order they are encoded.

[0087] In Figures 14(b) and 16(b), the first subblock in coding order does not contain the significance coefficient. In this case, the quantization parameter QpA is not coded in the first subblock. On the other hand, the second and third subblocks in coding order contain the significance coefficient. Therefore, the quantization parameter QpB is coded in the second subblock, and the quantization parameter QpC is coded in the third subblock. Furthermore, in Figure 16(b), the quantization parameter QpD is coded in the fourth subblock located in the lower right.

[0088] Here, we will explain, using Figure 15, how the quantization parameter is set for a subblock when the first subblock in the coding order does not contain a significance coefficient. In Figure 15, the meaning of the thick border lines and diagonal lines is the same as in Figure 14. In Figure 15(a), the ternary tree-divided subblocks are the same as in Figure 14(b). That is, the first subblock in the coding order does not contain a significance coefficient, while the second and third subblocks do. Furthermore, in Figure 15(b), the quadary tree-divided subblocks are the same as in Figure 16(b). That is, the first subblock in the coding order does not contain a significance coefficient, while the second, third, and fourth subblocks do. As mentioned above, since the first subblock does not contain a significance coefficient, the quantization parameter QpA is not coded. In this case, the quantization parameter of the subblock located to the left or above the subblock in question, or the subblock immediately preceding it in the coding order, is set as QpA for the quantization parameter of the subblock in question. In Figures 15(a) and 15(b), Qpabove indicates the quantization parameter of the subblock located above, and Qpleft indicates the quantization parameter of the subblock located to the left. Furthermore, Qpprev indicates the quantization parameter of the immediately preceding subblock in coding order. Note that the squares on which Qpabove, Qpleft, and Qpprev are written do not represent the shape of the subblock, but rather indicate the immediately preceding quantization parameter in top, left, and coding order, respectively.

[0089] In Figures 14(c) and 16(c), the second subblock in coding order does not contain the significance coefficient. In this case, the quantization parameter QpA is not encoded in the second subblock. At this time, the quantization parameter QpA from the first subblock, which is the preceding subblock in coding order, is set in the second subblock. On the other hand, in Figure 14(c), the first and third subblocks in coding order contain the significance coefficient. Therefore, the quantization parameter QpA is encoded in the first subblock, and the quantization parameter QpC is encoded in the third subblock. Also, in Figure 16(c), the first, third, and fourth subblocks in coding order contain the significance coefficient. Therefore, the quantization parameter QpA is encoded in the first subblock, QpB is encoded in the third subblock, and QpC is encoded in the fourth subblock.

[0090] In Figures 14(d) and 16(d), the first and second subblocks in coding order contain significance coefficients, but the third subblock does not. In this case, the quantization parameter QpB is not coded in the third subblock. At this time, the quantization parameter QpB of the second subblock, which is the preceding subblock in coding order, is set in the third subblock. Furthermore, in Figure 16(d), the quantization parameter QpC is coded in the fourth subblock.

[0091] In Figure 16(e), the first, second, and third subblocks in coding order contain significance coefficients, but the fourth subblock does not. In this case, the quantization parameter QpC is not coded in the fourth subblock. At this time, the quantization parameter QpC of the third subblock, which is the immediately preceding subblock in coding order, is set in the fourth subblock.

[0092] Thus, as a result of comparing the quantization control threshold with the number of pixels in each subblock, when quantization parameters are encoded in all of multiple subblocks, the quantization parameters are encoded in association with the subblock containing the significance coefficient. If a subblock does not contain a significance coefficient, the quantization parameters are not encoded in that subblock, but the quantization parameters of the immediately preceding subblock in the encoding order are set for that subblock. However, the quantization parameters set for subblocks that do not contain a significance coefficient are not limited to this; they may also be the quantization parameters of adjacent subblocks to the left or above, or the average value of the quantization parameters of multiple subblocks.

[0093] In the integrated encoding unit 111, information regarding the quantization control threshold is encoded. In this embodiment, the number of pixels in the subblock is encoded as information regarding the quantization control threshold. The number of pixels in the subblock is a positive integer; for example, in the partitioning example shown in Figure 9(b) or Figure 10(a), 512 is encoded as the quantization control threshold. However, in this embodiment, the information regarding the quantization control threshold is not limited to the number of pixels in the subblock. An index calculated based on the number of pixels in the subblock may also be encoded. Here, the numerical value to be encoded as the index will be explained using the table shown in Figure 21. However, the numerical value shown in Figure 21 is just an example. In this embodiment, when the number of pixels in the subblock can be expressed as a power of 2, the value of the exponent is encoded as the index. For example, when the number of pixels in the subblock is 512, the corresponding quantization control threshold is 512 = 2^9, so the numerical value representing the exponent of 2 shown in the second column of Figure 21, 9, is encoded. Furthermore, if the number of pixels in a subblock is 128, then 128 = 2^7, so 7 is encoded. On the other hand, if the ratio of each subblock to the number of pixels in the base block is calculated based on the number of pixels in the base block, and the reciprocal of that value can be expressed as a power of 2, then the value of that power of 2 exponent may be used as the index for encoding. For example, if the base block has 1024 (32 × 32) pixels and the quantization control threshold is 1024, then the ratio to the number of pixels in the base block is 1, and its reciprocal is also 1, so it is expressed as 1 = 2^0. In this case, 0 is encoded, as shown in the 5th column of Figure 21. Alternatively, if the quantization control threshold is 64, then the ratio to the number of pixels in the base block is 1 / 16, and its reciprocal is 16, so it is expressed as 16 = 2^4. In this case, 4 is encoded. This allows the quantization control threshold to be encoded with fewer bits, especially when encoding using Golomb coding, by encoding 0 when the quantization control threshold is at its maximum value (the same as the number of pixels in the basic block).

[0094] Furthermore, if the number of pixels in a subblock is expressed in combination with a power of 2 and a power of 3, it is also possible to encode two indices corresponding to both the power of 2 and the power of 3. For example, if the number of pixels in a subblock is 192, then 192 = 2^6 × 3^1, so two indices are encoded: the power of 2 m=6 shown in the second column of Figure 21 and the power of 3 n=1 shown in the third column. Also, if the number of pixels in a subblock is 80, then 80 = 2^6 × 5^1, so two indices are encoded: the power of 2 m=6 and the power of 5 k=1. Moreover, if the number of pixels in a subblock is 120, then 120 = 2^3 × 3^1 × 5^1. Therefore, three indices are encoded: the power of 2 m=3 shown in the second column of Figure 21, the power of 3 n=1 shown in the third column of the same figure, and the power of 5 k=1 shown in the fourth column of the same figure. This makes it possible to set quantization control thresholds that cannot be expressed as powers of 2, allowing for more precise control of quantization parameters.

[0095] Furthermore, in this embodiment, while it is stated that multiple corresponding indices are encoded for the number of pixels in a subblock and the exponent used to represent the quantization control threshold, the embodiment is not limited to this. For example, the index of the exponent m of 2 may be encoded as a required value, and flags may be encoded to indicate whether or not each of the other exponents are encoded. For example, if the quantization control threshold is calculated by multiplying a power of 2 value by a power of 3 value, such as 192, a flag representing the encoding of the exponent of 3 and the value of the exponent of 3 may be encoded. That is, in the case of 192, 1 may be used as the flag representing the encoding of the exponent of 3, 0 as the flag representing the non-encoding of the exponent of 5, 6 as the exponent of 2, and 1 as the exponent of 3, as shown in Figure 21. On the other hand, if the quantization control threshold can be calculated by multiplying a power of 2 value by a power of 5 value, such as 80, a flag representing the encoding of the exponent of 5 and the value of the exponent of 5 may be encoded. That is, in the case of 80, a flag of 0 representing the unencoded power of 3, a flag of 1 representing the encoded power of 5, a flag of 4 representing the power of 2, and a flag of 1 representing the power of 5 may be encoded. Furthermore, if the quantization control threshold can be calculated by multiplying by values ​​that are powers of 2, 3, or 5, such as 120, flags representing the encoding of each of the powers of 2, 3, and 5 may be encoded, along with the values ​​of the powers of 2, 3, and 5. That is, a flag of 1 representing the encoding of the power of 3, a flag of 1 representing the encoding of the power of 5, a flag of 3 representing the power of 2, a flag of 1 representing the power of 3, and a flag of 15 may be encoded. In this way, in addition to the index of the power of 2, only the indices necessary when the flag is true can be encoded.

[0096] In this embodiment, we have coded a flag indicating the existence of each exponent for prime numbers greater than or equal to 3 that constitute the quantization control threshold, but we are not limited to this. For example, we may code information representing the value of the largest prime number that constitutes the value of the quantization control threshold. That is, if the largest prime number is 5, counting prime numbers from smallest to largest, 5 is the 3rd, so we may code index 2 as information representing the value of the largest prime number that constitutes the value of the quantization control threshold. Also, if there are two prime numbers other than 2 that constitute the quantization control threshold, namely 3 and 11, the largest prime number is 11, and 11 is the 5th prime number from smallest to largest, so we may code index 4. In this way, the index is uniquely determined by counting prime numbers from smallest to largest. Furthermore, if the index is 0, it may mean that the exponent that should correspond to prime numbers greater than or equal to 3 is not used in the quantization control threshold. In this case, the value of the exponent that should correspond to prime numbers greater than or equal to 3 is 0. If the largest prime number is 5, in addition to index 2, we may code the values ​​of the exponents of 2, 3, and 5. For example, if the quantization control threshold is 720, then 4, 2, and 1 may be encoded, as shown in Figure 21. Similarly, if the quantization control threshold is 640, then 7, 0, and 1 may be encoded. In this way, by using information representing the value of the largest prime number that constitutes the quantization control threshold, any number of pixels can be efficiently encoded using a few power exponents.

[0097] Note that the numerical values ​​corresponding to powers of 2 are not limited to exponents of 2. For example, they may be derived from the reciprocal of the ratio of the number of pixels in the basic block. That is, if the size of the basic block is 1024 (32 × 32) pixels, the exponent of 2 corresponding to 64 pixels is 6, but the ratio to 1024 pixels is 1 / 16, and as shown in the 5th column of Figure 21, its reciprocal is 16 = 2^4. Therefore, when the quantization control threshold is 192, instead of 6, 4 may be encoded as the numerical value corresponding to the power of 2, and a flag 1 indicating the encoding of an exponent of 3 and the exponent of 3, 1, may be encoded. In this way, by using a value derived from the reciprocal of the ratio of the number of pixels in the basic block, the quantization control threshold can be represented with fewer bits than encoding the exponent.

[0098] In this embodiment, an index based on the value of a power exponent is encoded as the quantization control threshold, but this is not limited to this. An index assigned sequentially from the maximum number of pixels in a subblock to the minimum number of pixels may also be encoded. For example, as shown in Figure 21, if there are 19 different values ​​for the number of pixels in a subblock, with the maximum value being 1024 pixels and the minimum value being 8 pixels, index 0 may be assigned to 1024 pixels and index 18 to 8 pixels. In this case, 0 may be encoded when the quantization control threshold is 1024, and 2 may be encoded when it is 720. By encoding 0 when the quantization control threshold is at its maximum value (the same as the number of pixels in the base block), the quantization control threshold can be encoded with fewer bits, especially when encoding using Golomb coding or the like.

[0099] The method for encoding information regarding the quantization control threshold is not specifically defined, but Golomb coding, arithmetic coding, Huffman coding, etc., can be used. These codes and the coded data input from the coding unit 110 are multiplexed to form a bitstream. Finally, the bitstream is output to the outside from terminal 112.

[0100] Figure 6(a) shows an example of a bitstream containing encoded information about the quantization control threshold. The information about the quantization control threshold is included in either the sequence or picture header. In this embodiment, it is included in the picture header as shown in Figure 6(a). However, the encoding location is not limited to this, and it may also be included in the sequence header as shown in Figure 6(b).

[0101] Figure 3 is a flowchart showing the encoding process in the image encoding device according to this embodiment.

[0102] First, in step S301, the block division unit 102 divides the input image in frame units into basic block units.

[0103] In step S302, the generation unit 103 determines a quantization control threshold that indicates the unit for encoding the quantization parameters. This information is then used as quantization control threshold information. The quantization control threshold information is also encoded by the integrated encoding unit 111.

[0104] In step S303, the prediction unit 104 performs a division process on the basic block image data generated in step S301 to generate subblocks. The prediction unit 104 then performs a prediction process on each of the generated subblocks to generate prediction information such as block division and prediction mode, as well as predicted image data. Furthermore, it calculates a prediction error from the input image data and the predicted image data.

[0105] In step S304, the conversion / quantization unit 105 performs an orthogonal transformation on the prediction error calculated in step S303 to generate conversion coefficients. Furthermore, quantization is performed using the quantization parameters determined based on the quantization control threshold information generated in step S302, and residual coefficients are generated. Specifically, as described above, the quantization control threshold information is compared with the number of pixels in the subblock to determine whether or not the quantization parameters are shared among the subblocks within the basic block. Based on this determination, quantization of each subblock is performed using the quantization parameters corresponding to the subblocks in each region, and residual coefficients for each subblock are generated.

[0106] In step S305, the inverse quantization / inverse transformation unit 106 inversely quantizes and inversely orthogonal transforms the residual coefficients generated in step S304 to reconstruct the prediction error. The same quantization parameters used in step S304 are used for the inverse quantization process in this step.

[0107] In step S306, the image playback unit 107 plays back a predicted image based on the prediction information generated in step S303. Furthermore, it plays back image data from the played-back predicted image and the prediction error generated in step S305.

[0108] In step S307, the encoding unit 110 encodes the prediction information generated in step S303 and the residual coefficients generated in step S304, along with the block division information, to generate encoded data. It also encodes the quantization parameters used in step S304 based on the quantization control threshold information generated in step S302. Furthermore, it generates a bitstream including other encoded data. Specifically, in the region where the quantization parameters determined in step S304 are shared, the quantization parameters are encoded in association with subblocks containing at least one significance coefficient in the order of the subblocks to be encoded.

[0109] In step S308, the control unit 100 of the image encoding device determines whether the encoding of all basic blocks in the frame has been completed. If it has been completed, it proceeds to step S309; ​​otherwise, it targets the next basic block and returns to step S303.

[0110] In step S309, the in-loop filter unit 109 performs in-loop filtering on the image data reproduced in step S306, generates a filtered image, and then terminates the process.

[0111] In summary, by generating quantization control threshold information in step S302, and performing quantization and encoding processing based on this information in steps S304 and S307, it is possible to appropriately encode the quantization parameters. As a result, it is possible to improve the image quality of the encoded image while reducing the total amount of data in the generated bitstream.

[0112] In this embodiment, it was explained that the number of pixels in each subblock is compared with one quantization control threshold to determine whether or not quantization parameters should be shared among multiple subblocks, but this is not limited to this. Two or more quantization control thresholds may be used. Here, the case in which quantization parameters are encoded using two quantization control thresholds will be explained using Figure 10(e).

[0113] In the cases of Figures 10(c) and 10(d) described above, the quantization control threshold is compared with the subblock with the largest number of pixels among the multiple subblocks generated by performing an asymmetric partition once. The quantization control threshold compared with the subblock with the largest number of pixels in this way is called the first quantization control threshold (hereinafter referred to as the first threshold). In contrast, the quantization control threshold compared with the subblock with the smallest number of pixels among the multiple subblocks generated by performing an asymmetric partition once is called the second quantization control threshold (hereinafter referred to as the second threshold). In the case of Figure 10(e), the first threshold is 128 and the second threshold is 32. When using the first and second thresholds, the upper left region on the left side of Figure 10(e) is divided into four subblocks by a single asymmetric quadtree partition, and the lower left subblock with the largest number of pixels has 144 pixels, which is greater than or equal to the first threshold. However, at the same time, the upper right subblock with the smallest number of pixels has 16 pixels, which is less than the second threshold. Therefore, since the comparison conditions for the first and second thresholds are not simultaneously met, the four subblocks share one quantization parameter, as shown on the right side of Figure 10(e). Next, in the upper right and lower left regions of the left side of Figure 10(e), the area is divided into two subblocks by an asymmetric binary tree partition. Of these, the larger subblock located on the upper side in the upper right region, and the larger subblock located on the left side in the lower left region, have 192 pixels, which is greater than or equal to the first threshold of 128. At the same time, the smaller subblock located on the lower side in the upper right region, and the smaller subblock located on the right side in the lower left region, have 64 pixels, which is greater than or equal to the second threshold of 32. Therefore, in this case, the quantization parameter is encoded in each of the two subblocks.

[0114] Furthermore, in the lower right region, as in Figure 10(c) above, two asymmetric binary tree partitions are performed. In the first binary tree partition, the larger subblock on the right has 192 pixels, which is greater than or equal to the first threshold. At the same time, the smaller subblock on the left has 64 pixels, which is greater than or equal to the second threshold of 32. In the second binary tree partition, which targets the larger subblock on the right generated in the first partition, the larger subblock on the upper side has 144 pixels, which is greater than or equal to the first threshold. At the same time, the smaller subblock on the lower side has 48 pixels, which is greater than or equal to the second threshold. Therefore, in the lower right region of Figure 10(e), the quantization parameters are encoded for each of the three subblocks. In this way, by using two quantization control thresholds, the quantization parameters are shared within the right-hand block unit of Figure 10(e). Furthermore, within that block unit, one quantization parameter is encoded in association with the subblock that contains the significance coefficient first in the encoding order. In this case, eight quantization parameters are encoded, as shown on the right side of Figure 10(e). In contrast, Figure 10(d) uses only the first threshold for comparison, and the threshold value is 128, the same as in Figure 10(e). In this case, eleven quantization parameters are encoded, as shown on the right side of Figure 10(d). Therefore, in the case of Figure 10(e), which uses two quantization control thresholds, three fewer quantization parameters are encoded compared to the case of Figure 10(d), which uses one quantization control threshold. Thus, by using a second threshold in addition to the first threshold, the number of quantization parameters to be encoded can be set more flexibly compared to the case where only one quantization control threshold is used, and excessive encoding of quantization parameters can be suppressed.

[0115] It was explained that quantization parameters are encoded in each subblock generated by the asymmetric partition when both the comparison using the first quantization control threshold and the comparison using the second quantization control threshold are valid. However, this embodiment is not limited to this. For example, quantization parameters may be encoded in each subblock when at least one comparison is valid.

[0116] Let's use Figure 10(c) to explain a specific example. In Figure 10(c), let's assume the first threshold is 192 and the second threshold is 32. In that case, in the upper left region on the left side of Figure 10(c), the largest subblock, 144 pixels, is less than the first threshold, and the smallest subblock, 16 pixels, is less than the second threshold. Therefore, since no comparisons are met, we can encode one quantization parameter in the upper left region, as shown on the right side of Figure 10(c). In the upper right and lower left regions of Figure 10(c), the largest subblock, 192 pixels, is greater than or equal to the first threshold, and the smallest subblock, 64 pixels, is greater than or equal to the second threshold. Therefore, since one or more comparisons are met, we can encode two quantization parameters in the upper right and lower left regions, as shown on the right side of Figure 10(c). In the lower right region on the left side of Figure 10(c), we perform an asymmetric binary tree partition twice. In the first binary tree split, the larger subblock on the right has 192 pixels, which is greater than or equal to the first threshold, and the smaller subblock on the left has 64 pixels, which is greater than or equal to the second threshold. Furthermore, in the second binary tree split targeting the larger subblock on the right generated in the first split, the larger subblock on the upper side has 144 pixels, which is less than the first threshold of 192. Also, the smaller subblock on the lower side has 48 pixels, which is greater than or equal to the second threshold of 32. Therefore, the comparison using the second threshold is valid, and the lower right region on the left side of Figure 10(c) may have three quantization parameters encoded, as shown in the lower right region on the right side of Figure 10(d). Thus, if only the first threshold is used, seven quantization parameters are encoded, and if at least one of the first and second thresholds is valid, eight quantization parameters may be encoded.

[0117] Thus, if at least one comparison is valid, it becomes possible to encode quantization parameters into more subblocks, enabling fine-grained image quality adjustments at the subblock level.

[0118] Here, the encoding of the two quantization control thresholds will be explained. In this embodiment, the number of pixels in a subblock is encoded as information regarding the quantization control thresholds. The number of pixels in a subblock is a positive integer. For example, in the partitioning example shown in Figure 10(e), 128 is encoded as the first threshold, and 32 is encoded as the second threshold. The encoding of the second threshold may be a ratio with respect to the first threshold. For example, if the first and second thresholds are 128 and 32, the encoding may be 128 and 4. Furthermore, the encoding of the second threshold may be the difference between it and the first threshold. For example, if the first and second thresholds are 192 and 128, 192 and 64 can also be encoded.

[0119] In this embodiment, information regarding the quantization control threshold is not limited to the number of pixels in the subblock. An index calculated based on the number of pixels in the subblock may be encoded. Here, the numerical values ​​to be encoded as the index will be explained using the table shown in Figure 21. However, the numerical values ​​shown in Figure 21 are just examples. In this embodiment, if the number of pixels in the subblock can be expressed as a power of 2, the value of the exponent may be encoded as the index. For example, the first threshold corresponding to the case where the number of pixels in the subblock is 128 is 128 = 2^7, so the numerical value representing the exponent of 2 shown in the second column of Figure 21, which is 7, may be encoded. Furthermore, the second threshold corresponding to the case where the number of pixels in the subblock is 32 is 32 = 2^5, so 5 may be encoded. Also, if the second threshold is expressed as the difference between it and the first threshold, 2 may be encoded. Alternatively, if the number of pixels in the subblock is 256, 8 may be encoded as the first threshold, and if the number of pixels in the subblock is 16, 4 may be encoded as the second threshold. In this case, since the first threshold is twice the second threshold, we can encode 2 instead of 8.

[0120] Furthermore, if the number of pixels in a subblock is expressed in combination with powers of 2 and powers of 3, two indices corresponding to both powers of 2 and powers of 3 may be encoded for the first and second thresholds, respectively. For example, if the number of pixels corresponding to the first threshold is 144, then 144 = 2^4 × 3^2, so two indices are encoded: the power of 2 m=4 shown in the second column of Figure 21 and the power of 3 n=2 shown in the third column. Additionally, if the number of pixels corresponding to the second threshold is 10, then 10 = 2^1 × 5^1, so the power of 2 m=1 and the power of 5 k=1 may be encoded as two indices. That is, 6 and 1 may be encoded as the first threshold, and 2 and 5 as the second threshold, so a total of four indices may be encoded.

[0121] In this embodiment, image data is input frame by frame, and an encoding process is performed to generate and output a bitstream. However, the encoding process is not limited to image data. For example, feature data used in machine learning, such as object recognition, can be input in a two-dimensional shape, and an encoding process can be performed to encode the bitstream. This makes it possible to efficiently encode feature data used in machine learning.

[0122] In this embodiment, the number of pixels in a subblock is compared with the quantization control threshold, but this is not limited to this. Any information regarding the number of pixels in a subblock is acceptable. Instead of the number of pixels in a subblock, the logarithmic value of the number of pixels in a subblock, the width of the subblock, or the height of the subblock may be used. The quantization control threshold is changed to match the information regarding the number of pixels in a subblock used for comparison.

[0123] <Embodiment 2> Figure 2 is a block diagram showing the configuration of the image decoding device. In this embodiment, the decoding of encoded data generated in the image encoding device shown in Figure 1 will be explained as an example.

[0124] Terminal 201 is an input terminal that receives an encoded bitstream.

[0125] The separation / decoding unit 202 separates the bitstream into coded data related to the decoding process and residual coefficients, and also decodes the coded data present in the header portion of the bitstream. In this embodiment, it decodes the quantization control threshold information and outputs it to the next stage. The separation / decoding unit 202 operates in the reverse direction of the integrated encoding unit 111 in Figure 1.

[0126] The decoding unit 203 obtains residual coefficients and prediction information from the coded data output from the separation and decoding unit 202.

[0127] The inverse quantization / inverse transformation unit 204 performs inverse quantization on the residual coefficients input in block units, and then performs an inverse orthogonal transformation to obtain the prediction error. In this embodiment, the inverse quantization / inverse transformation unit 204 performs inverse quantization and inverse transformation, but it may also be configured to be divided into an inverse quantization unit that performs inverse quantization and an inverse transformation unit that performs inverse transformation.

[0128] Frame memory 206 is memory that stores the image data of the played-back picture.

[0129] The image playback unit 205 generates predicted image data by appropriately referring to the frame memory 206 based on the input prediction information. Then, it generates and outputs reproduced image data from this predicted image data and the prediction error reproduced by the inverse quantization / inverse transform unit 204.

[0130] The in-loop filter section, 207, is the same as the in-loop filter section 109 in Figure 1. It performs in-loop filtering, such as deblocking filtering, on the regenerated image and outputs the filtered image.

[0131] Terminal 208 is an output terminal that outputs the reproduced image data to an external device.

[0132] The image decoding operation in the above-described image decoding device is explained below. In this embodiment, the bitstream generated in this embodiment is decoded.

[0133] In Figure 2, the control unit 200 is a processor that controls the entire image decoding device, and the bitstream input from terminal 201 is input to the separation and decoding unit 202. The separation and decoding unit 202 separates the bitstream into coded data related to decoding processing and coefficients, and decodes the coded data present in the header portion of the bitstream. Specifically, it decodes the quantization control threshold information. In this embodiment, first, the quantization control threshold information is decoded from the picture header of the bitstream shown in Figure 6(a). The quantization control threshold information obtained in this way is output to the decoding unit 203 and the inverse quantization / inverse transformation unit 204. Furthermore, coded data in block units of the picture data is output to the decoding unit 203.

[0134] The decoding unit 203 decodes the coded data and obtains residual coefficients, prediction information, and quantization parameters. The residual coefficients and quantization parameters are output to the inverse quantization / inverse transform unit 204, and the obtained prediction information is output to the image playback unit 205. In decoding the quantization parameters that constitute the quantization information, an identifier indicating the difference between the quantization parameter of the subblock to be decoded and the predicted value calculated using the quantization parameters of subblocks decoded before that subblock is decoded. In this embodiment, the quantization parameter encoded immediately before the subblock in the decoding order is used as the predicted value, and the difference between that and the quantization parameter of the subblock is calculated, but the predicted value of the quantization parameter is not limited to this. The quantization parameter of the subblock adjacent to the left or above the subblock may be used as the predicted value, or a value calculated from the quantization parameters of multiple subblocks, such as the average value, may be used as the predicted value. Furthermore, if the subblock to be processed is the first subblock in the decoding order among the subblocks belonging to the first basic block in the basic block row, the quantization parameter of the subblock in the basic block directly above it may be used as the predicted value. This enables parallel processing on a basic block row basis. The first basic block in a basic block row refers to the basic block that has a picture boundary or tile boundary on its left side.

[0135] Here, the calculation of subblock size is explained using Figure 8. Here, the subblock size is the number of pixels of the subblock corresponding to the division ratio of the block being divided. Figure 8 shows examples of subblock division types when a 32x32 pixel base block is divided into subblocks. The numbers written in each subblock shown in Figures 8(a) to (k) represent the number of pixels of each subblock. Figure 8(a) shows that the base block has never been divided and the number of pixels of the subblock is the same as the base block, 1024 (32x32). Figure 8(b) shows that the base block has been quad-divided and the number of pixels of each subblock is 256. Figures 8(c) and (d) show that the base block has been binary-divided and the number of pixels of each subblock is 512. Figures 8(e) and (f) show that the base block has been ternarily-divided. If the number of pixels of the subblock after division is halved, it will be 512 pixels. On the other hand, if the number of pixels in the sub-block after division is one-quarter, it becomes 256 pixels. Furthermore, Figures 8(g) to 8(k) show asymmetric divisions. For example, Figure 8(g) is divided into a vertical sub-block of 256 (8×32) pixels and a vertical sub-block of 768 (24×32) pixels, so the division ratio is 1:3. In Figure 8(h), the division ratio is 3:1, and the number of pixels in the left sub-block exceeds the number of pixels in the right sub-block. Also, in Figure 8(i), the top left is divided into a horizontal sub-block of 192 (24×8) pixels, the top right into a square sub-block of 64 (8×8) pixels, the bottom left into a square sub-block of 576 (24×24) pixels, and the bottom right into a vertical sub-block of 192 (8×24) pixels. Figure 8(j) shows an upper subblock with 640 (32×20) pixels and a lower subblock with 384 (32×12) pixels. Figure 8(k) is divided vertically into four sections, with the first subblock from the top having 128 (32×4) pixels, the second subblock having 512 (32×16) pixels, the third subblock having 256 (32×8) pixels, and the fourth subblock having 128 (32×4) pixels.In the asymmetric partitioning shown in Figures 8(g) to (k), in the multiple subblocks generated in a single partitioning, there is a subblock with the largest number of pixels and a subblock with the smallest number of pixels. For example, in Figure 8(i), the subblock in the lower left has the largest number of pixels, and the subblock in the upper right has the smallest number of pixels.

[0136] Here, we will explain how the quantization parameters are decoded according to the quantization control threshold and the number of pixels in the subblocks, using Figure 9. In Figures 9(a) to (e), the square enclosed by the outermost frame represents the basic block. In each of Figures 9(a) to (e), the left side shows the subblock division, and the right side shows the region where the quantization parameters are shared. The numbers within each block in the figures represent the number of pixels in each subblock, which are 256 pixels, 128 pixels, and 64 pixels in descending order of size. Qp represents the quantization parameter. The decoding of quantization parameters when using asymmetric division will be described later using Figure 10. Figures 9(a), (b), (c), (d), and (e) show the cases where the quantization control threshold is 1024 pixels, 512 pixels, 256 pixels, 128 pixels, and 64 pixels, respectively. In the case of Figure 9(a), i.e., when the quantization control threshold is 1024 pixels, a common quantization parameter is used for all subblocks in the figure, and the quantization parameter is decoded in association with the subblock containing the first significance coefficient in the decoding order. In this case, only one quantization parameter is decoded.

[0137] The significance coefficient refers to the non-zero residual coefficients after transformation and quantization. In other words, the presence of a significance coefficient indicates that at least one non-zero residual coefficient exists in the subblock obtained by decoding the coded data. The relationship between the presence or absence of a significance coefficient and the decoding of quantization parameters will be described later using Figures 11 and 12.

[0138] In the case of Figure 9(b), that is, when the quantization control threshold is 512 pixels, there are no subblocks with more than 512 pixels, so the result is the same as in Figure 9(a).

[0139] In the case of Figure 9(c), that is, when the quantization control threshold is 256 pixels, the quantization parameters are shared within the right-hand block of Figure 9(c). Furthermore, within that block, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order. In this case, four quantization parameters are decoded.

[0140] In the case of Figure 9(d), that is, when the quantization control threshold is 128 pixels, the quantization parameters are shared within the right-hand block of the figure, and one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order within that unit. Here, we will explain the decoding of quantization parameters in terminally partitioned subblocks. In Figure 9(d), in the terminally partitioned subblock located in the lower left, the number of pixels differs between the subblocks at both ends and the central subblock. Specifically, the two subblocks that are divided into quarters of the number of pixels have 64 pixels each, while the central subblock, which is divided into half the number of pixels, has 128 pixels. On the other hand, the value of the quantization control threshold is 128 pixels. In this case, since the number of pixels in the subblock with the largest number of pixels among the terminally partitioned subblocks is greater than or equal to the quantization control threshold, the quantization parameters are decoded for each of the three subblocks. Thus, in Figure 9(d), there are 7 quantization parameters that are decoded.

[0141] In the case of Figure 9(e), i.e., when the quantization control threshold is 64 pixels, the quantization parameters are shared within the right-hand block of the figure, and the quantization parameters are decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order within that unit. In this case, 10 quantization parameters are decoded.

[0142] Here, the decoding of quantization parameters when using an asymmetric partition is explained using Figure 10. The thick lines and the rectangles within the thick lines in Figure 10 are the same as in Figure 9, so their explanation is omitted. The numbers in Figure 10 indicate the number of pixels in each subblock. Figures 10(a), (b), (c), and (d) show the cases where the quantization control threshold is 512 pixels, 256 pixels, 192 pixels, and 128 pixels, respectively. For clarity, in Figure 10, the quad partition in Figure 8(b) is used for the first partition of the recursive partition, dividing the basic block into four 16x16 pixel blocks, and asymmetric partitions are used for subsequent partitions. The four subblocks generated using the quad partition in Figure 8(b) are referred to here as the upper left region, upper right region, lower left region, and lower right region, based on their position in the basic block. The upper left region generates four subblocks in one partition using the quad tree partition in Figure 8(i). In the upper right and lower left regions, two subblocks are generated by asymmetric binary tree partitioning. Then, in the lower right region, three subblocks are generated by performing asymmetric binary tree partitioning twice.

[0143] In Figure 10(a), the quantization control threshold is 512 pixels, and the number of pixels in all subblocks in the figure is less than the quantization control threshold. Therefore, a common quantization parameter is used for all subblocks in the figure, and the quantization parameter is decoded in association with the subblock containing the first significance coefficient in the decoding order. In this case, only one quantization parameter is decoded.

[0144] In Figure 10(b), the quantization control threshold is 256 pixels. In the left side of Figure 10(b), the basic block is divided once using an asymmetric quadtree partition for the initial division, generating four subblocks. In this case, the four subblocks have 256 (16 × 16) pixels, which is greater than or equal to the quantization control threshold. Therefore, one quantization parameter is decoded for the upper left, upper right, lower left, and lower right regions. Next, the upper left region is further divided once using an asymmetric quadtree partition. At this time, if there are subblocks with fewer pixels than the quantization control threshold, the subblock with the largest number of pixels is compared with the quantization control threshold. In the upper left region, the largest number of pixels is 144, and the quantization control threshold is 256. Therefore, the four subblocks share the quantization parameter. The upper right and lower left regions are then divided into two subblocks of 192 pixels and 64 pixels using an asymmetric binary tree partition. However, since the number of pixels in each subblock is less than the quantization control threshold, the two subblocks in the upper right and lower left regions share the quantization parameters. In the lower right region, the 16x16 pixel subblock is first divided horizontally in a 1:3 ratio, generating a 64-pixel subblock and a 192-pixel subblock. Next, the 192-pixel subblock is further divided vertically in a 3:1 ratio, generating a 144-pixel subblock and a 48-pixel subblock. Meanwhile, the quantization control threshold is 256 pixels. In this case, of the two subblocks generated in the first binary tree division, the larger subblock on the right has 192 pixels, which is less than the quantization control threshold. Furthermore, of the two subblocks generated in the second binary tree division, the larger subblock on the top has 144 pixels, which is also less than the quantization control threshold. Therefore, the three subblocks share the quantization parameters. In this case, the quantization parameters are shared at the unit of the right-hand block in Figure 10(b). Furthermore, for each block, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order. Similar to the example shown in Figure 9(c), in this case, four quantization parameters are decoded.

[0145] In Figure 10(c), the quantization control threshold is 192 pixels. Here, we will explain the decoding of quantization parameters when the subblock is divided into four parts using an asymmetric partition. On the left side of Figure 10(c), the upper left region is divided into four subblocks by a single asymmetric quadtree partition, each with a different number of pixels. Of the four subblocks, the one located in the lower left has 144 pixels and is the subblock with the most pixels. The subblock located in the upper right has 16 pixels and is the subblock with the fewest pixels. The subblocks located in the upper left and lower right each have 48 pixels. In contrast, the quantization control threshold is 192 pixels. Therefore, since the number of pixels in the subblock with the most pixels, 144 pixels, is less than the quantization control threshold, the four subblocks share the quantization parameters.

[0146] Here, we will further explain the decoding of quantization parameters when a subblock is divided into two using an asymmetric partition. In the case of an asymmetric binary tree partition, the subblock with the largest number of pixels among the two subblocks generated by performing the partition once is compared with the quantization control threshold. If the number of pixels of the subblock with the largest number of pixels is greater than or equal to the quantization control threshold, the quantization parameters are decoded into each subblock generated by the asymmetric partition. If the number of pixels of the subblock with the largest number of pixels is less than the quantization control threshold, the two subblocks generated by the asymmetric partition share the quantization parameters. For example, on the left side of Figure 10(c), the asymmetric binary tree partitioned subblocks located in the upper right and lower left have different numbers of pixels. In the upper right region, a 16x16 pixel subblock is binary tree partitioned vertically in a 3:1 ratio, while in the lower left region, a binary tree partition is performed horizontally in a 1:3 ratio. The two subblocks after the split are as follows: the larger subblock on the left has 192 pixels, and the smaller subblock on the right has 64 pixels. Meanwhile, the quantization control threshold is 192 pixels. In this case, since the larger subblock generated by the asymmetric binary tree split has 192 pixels and is above the quantization control threshold, the quantization parameters are decoded in each of the two subblocks. Also, on the left side of Figure 10(c), the lower right region is divided into three subblocks by repeating the asymmetric binary tree split twice. Specifically, a 16x16 pixel subblock is first split horizontally in a 1:3 ratio, generating a 64-pixel subblock and a 192-pixel subblock. Next, the 192-pixel subblock is further split vertically in a 3:1 ratio, generating a 144-pixel subblock and a 48-pixel subblock. Meanwhile, the quantization control threshold is 192 pixels. In this case, of the two subblocks generated in the first binary tree split, the larger subblock located on the right has 192 pixels, which is above the quantization control threshold. Therefore, the quantization parameters are decoded for each of the two subblocks.However, of the two subblocks generated in the second binary tree split, the larger subblock located on the upper side has 144 pixels, which is less than the quantization control threshold. Therefore, in this case, the two subblocks share one quantization parameter, as shown in the lower right block on the right side of Figure 10(c). Furthermore, for each block, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order. In this case, seven quantization parameters are decoded. Thus, if the number of pixels in a subblock compared to the quantization control threshold is less than the quantization control threshold, the target block from which to decode the quantization parameters is changed.

[0147] In Figure 10(d), the quantization control threshold is 128 pixels. Here, we will explain the decoding of quantization parameters when the subblock is divided into four parts using an asymmetric partition. On the left side of Figure 10(d), the upper left region is divided into four subblocks by one asymmetric quadtree partition, each with a different number of pixels. Of the four subblocks, the one located in the lower left has 144 pixels and is the subblock with the most pixels. The subblock located in the upper right has 16 pixels and is the subblock with the fewest pixels. The subblocks located in the upper left and lower right each have 48 pixels. On the other hand, the value of the quantization control threshold is 128 pixels. In this case, the subblock with the most pixels has 128 pixels, which is greater than or equal to the quantization control threshold, so the quantization parameters are decoded for each of the four subblocks. Next, on the left side of Figure 10(d), the lower right region is subjected to two asymmetric binary tree partitions. In the first binary tree split, a 16x16 pixel subblock is first split horizontally in a 1:3 ratio, generating a 64-pixel subblock and a 192-pixel subblock. In the subsequent second binary tree split, the 192-pixel subblock on the right is further split vertically in a 3:1 ratio, generating a 144-pixel subblock and a 48-pixel subblock. Meanwhile, the quantization control threshold is 128 pixels. Therefore, of the two subblocks generated in the second binary tree split, the larger upper subblock has 144 pixels, which is greater than or equal to the quantization control threshold. Thus, in the lower right region, the quantization parameters are decoded for each of the three subblocks. In this case, the quantization parameters are shared within the right-hand block unit in Figure 10(d). Furthermore, within that block unit, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order. In this case, the quantization parameters that are decoded are 11, as shown on the right side of Figure 10(d).

[0148] In this manner, asymmetric partitioning is used, and if there are subblocks with fewer pixels than the quantization control threshold, the quantization control threshold is compared with the subblock with the largest number of pixels. If the number of pixels of a subblock compared with the quantization control threshold is less than the quantization control threshold, quantization parameters are assigned to each subblock. If the number of pixels of a subblock compared with the quantization control threshold is equal to or greater than the quantization control threshold, the quantization parameters are shared.

[0149] Here, we will explain the decoding of quantization parameters when asymmetric binary tree partitioning is performed multiple times and the subblocks are partitioned to have the same shape as a single ternary partition, using Figure 17. The thick lines and the rectangles within the thick lines in Figure 17 are the same as in Figure 9, so we will omit their explanation. The numbers in Figure 17 indicate the number of pixels in each subblock. Figures 17(a) to (d) show the cases where the quantization control thresholds are 1024, 768, 768, and 512, respectively. In the case of Figure 17(a), the basic block is first partitioned horizontally in a 1:3 ratio binary tree. Of the two subblocks generated by the partition, the larger subblock on the right has 768 pixels, which is less than the quantization control threshold of 1024. Next, the larger subblock on the right obtained in the first partition is partitioned horizontally in a 2:1 ratio in the second binary tree partition. As a result, of the two subblocks generated in the second split, the larger subblock located on the left has 512 pixels, which is below the quantization control threshold. Therefore, as shown in the right-hand diagram of Figure 17(a), only one quantization parameter is decoded. Alternatively, the first binary tree split can be performed by dividing the horizontal direction in a 3:1 ratio to generate the three subblocks shown in Figure 17(a). In this case, the larger subblock located on the left has 768 pixels, which is below the quantization control threshold of 1024. Subsequently, the larger subblock located on the left obtained in the first split is divided horizontally in a 1:2 ratio in the second binary tree split. As a result, of the two subblocks generated in the second split, the larger subblock located on the right has 512 pixels, which is below the quantization control threshold. Therefore, in the case of Figure 17(a), regardless of the horizontal splitting ratio in the first binary tree split, only one quantization parameter is decoded, as shown in the right-hand diagram of Figure 17(a).

[0150] In Figure 17(b), the first asymmetric binary tree partition divides the subblock horizontally in a 1:3 ratio. The larger subblock on the right side of the resulting subblock has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Subsequently, the larger subblock on the right side generated in the first partition is divided horizontally in a 2:1 ratio by a second asymmetric binary tree partition. The larger subblock on the left side of the resulting subblock has 512 pixels, which is less than the quantization control threshold. Therefore, the subblock generated in the second partition has a common quantization parameter decoded, as shown in the right-hand diagram of Figure 17(b). In other words, in this case, two quantization parameters are decoded for the basic block.

[0151] In the case of Figure 17(c), the first asymmetric binary tree partition is performed to divide the tree horizontally in a 3:1 ratio. The larger subblock on the left side of the resulting subblock has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Furthermore, the larger subblock on the left side generated in the first partition is divided horizontally in a 1:2 ratio by the second asymmetric binary tree partition. The larger subblock on the right side of the resulting subblock has 512 pixels, which is less than the quantization control threshold. Therefore, the subblock generated in the second partition is decoded with a common quantization parameter, as shown in the right-hand diagram of Figure 17(c). In other words, in this case, two quantization parameters are decoded for the basic block.

[0152] In both Figure 17(b) and Figure 17(c), the quantization control threshold is 768, and two quantization parameters are decoded for the basic block. However, of the three subblocks, in Figure 17(b), the central and right subblocks share the quantization parameter, while in Figure 17(c), the left and central subblocks share the quantization parameter.

[0153] In the case of Figure 17(d), the first asymmetric binary tree partition is performed to divide the base block horizontally in a 1:3 ratio. The larger subblock on the right side generated by the partition has 768 pixels, which is above the quantization control threshold. Furthermore, the larger subblock generated by the first asymmetric binary tree partition is divided in a 2:1 ratio by the second asymmetric binary tree partition. In this case, the larger subblock on the left side of the two subblocks generated by the binary tree partition has 512 pixels, which is above the quantization control threshold. Therefore, the three subblocks on the left side of Figure 17(d) decode the quantization parameters for each subblock, as shown in the right side of Figure 17(d). That is, three quantization parameters are decoded for the base block. Note that the base block may also be divided horizontally in a 3:1 ratio by the first asymmetric binary tree partition. Subsequently, the larger subblock on the right side generated by the first partition may also be divided in a 1:2 ratio by the second asymmetric binary tree partition. Even in this case, as shown in the right-hand diagram of Figure 17(d), three quantization parameters are decoded for the basic block.

[0154] Here, we will explain the decoding of quantization parameters when multiple asymmetric binary tree partitions are performed so that the partition shape is the same as that of a single asymmetric quadtree partition, using Figure 18. The thick lines and the rectangles within the thick lines in Figure 18 are the same as in Figure 9, so we will omit their explanation. The numbers in Figure 18 indicate the number of pixels in each subblock. The arrows shown in the right-hand diagrams of Figures 18(a) to (d) indicate the decoding order. In Figures 18(a) and (b), the quantization control threshold is 768, and in Figures 18(c) and (d), the quantization control threshold is 512. Also, in Figures 18(a) and (c), the first partition is a vertically asymmetric binary tree partition, and the subsequent second partition is a horizontally asymmetric binary tree partition. In contrast, in Figures 18(b) and (d), the first partition is a horizontally asymmetric binary tree partition, and the subsequent second partition is a vertically asymmetric binary tree partition.

[0155] In the case of Figure 18(a), first, a flag of 1 is decoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 1 is decoded to indicate that it is a vertical partition, and a flag of 1 is decoded to indicate that the partition ratio is 1:3. Then, as the first partition, an asymmetric binary tree partition is performed, dividing the base block into two subblocks in a vertical ratio of 1:3. In this case, the larger subblock located at the bottom has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Therefore, the smaller subblock located at the top and the larger subblock located at the bottom do not share quantization parameters. Subsequently, for the smaller subblock located at the top generated in the first partition, a flag of 0 is decoded to indicate that it is a horizontal partition, and a flag of 0 is decoded to indicate that the partition ratio is 3:1. Then, as the second partition, an asymmetric binary tree partition is performed, dividing the smaller subblock located at the top generated in the first partition into two subblocks in a horizontal ratio of 3:1. In this case, the larger subblock on the left, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 768. Therefore, the 192-pixel subblock on the left and the 64-pixel subblock on the right share the quantization parameters, and one quantization parameter is decoded. Additionally, for the larger subblock on the lower side, generated in the first split, 0 is decoded as a flag indicating a horizontal split, and 0 is decoded as a flag indicating a split ratio of 3:1. Then, as the second split, an asymmetric binary tree split is performed, dividing the larger subblock on the lower side, generated in the first split, into two subblocks horizontally in a 3:1 ratio. In this case, the larger subblock on the left, generated in the second split, has 576 pixels, which is less than the quantization control threshold of 768. Therefore, the 576-pixel subblock on the left and the 192-pixel subblock on the right share the quantization parameters, and one quantization parameter is decoded. Therefore, in this case, two quantization parameters are decoded for the basic block, as shown in the central diagram of Figure 18(a).Furthermore, the four subblocks generated by the partition are decoded in the order shown in the right-hand diagram of Figure 18(a). At this time, for each block unit that shares a quantization parameter, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order.

[0156] In the case of Figure 18(b), first, a flag of 1 is decoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 0 is decoded to indicate that it is a horizontal partition, and a flag of 0 is decoded to indicate that the partition ratio is 3:1. Then, as the first partition, an asymmetric binary tree partition is performed, dividing the base block into two subblocks horizontally in a ratio of 3:1. In this case, the larger subblock on the left has 768 pixels, which is greater than or equal to the quantization control threshold of 768. Therefore, the larger subblock on the left and the smaller subblock on the right do not share quantization parameters. Subsequently, for the larger subblock on the left generated in the first partition, a flag of 1 is decoded to indicate that it is a vertical partition, and a flag of 1 is decoded to indicate that the partition ratio is 1:3. Then, as the second partition, an asymmetric binary tree partition is performed, dividing the larger subblock on the left generated in the first partition vertically into two subblocks in a ratio of 1:3. In this case, the larger subblock located on the lower side, generated in the second split, has 576 pixels, which is less than the quantization control threshold of 768. Therefore, the 192-pixel subblock located on the upper side and the 576-pixel subblock located on the lower side share the quantization parameters, and one quantization parameter is decoded. Also, for the smaller subblock located on the right side, generated in the first split, 1 is decoded as a flag indicating that it is a vertical split, and 1 is decoded as a flag indicating that the split ratio is 1:3. Then, as the second split, an asymmetric binary tree split is performed, dividing the smaller subblock located on the right side, generated in the first split, into two subblocks in a vertical ratio of 1:3. In this case, the larger subblock located on the lower side, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 768. Therefore, the 64-pixel subblock located on the upper side and the 192-pixel subblock located on the lower side share the quantization parameters, and one quantization parameter is decoded. Therefore, in this case, two quantization parameters are decoded for the basic block, as shown in the central diagram of Figure 18(b).Furthermore, the four subblocks generated by the partition are decoded in the order shown in the right-hand diagram of Figure 18(b). At this time, for each block unit that shares a quantization parameter, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order.

[0157] In the case of Figure 18(c), first, a flag of 1 is decoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 1 is decoded to indicate that it is a vertical partition, and a flag of 1 is decoded to indicate that the partition ratio is 1:3. Then, as the first partition, an asymmetric binary tree partition is performed, dividing the base block into two subblocks in a vertical ratio of 1:3. In this case, the larger subblock located at the bottom has 768 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the smaller subblock located at the top and the larger subblock located at the bottom do not share quantization parameters. Subsequently, for the smaller subblock located at the top generated in the first partition, a flag of 0 is decoded to indicate that it is a horizontal partition, and a flag of 0 is decoded to indicate that the partition ratio is 3:1. Then, as the second partition, an asymmetric binary tree partition is performed, dividing the smaller subblock located at the top generated in the first partition into two subblocks in a horizontal ratio of 3:1. In this case, the larger subblock on the left, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 512. Therefore, the 192-pixel subblock on the left and the 64-pixel subblock on the right share the quantization parameters, and one quantization parameter is decoded. Additionally, for the larger subblock on the lower side, generated in the first split, 0 is decoded as a flag indicating a horizontal split, and 0 is decoded as a flag indicating a split ratio of 3:1. Then, as the second split, an asymmetric binary tree split is performed, dividing the larger subblock on the lower side, generated in the first split, into two subblocks horizontally in a 3:1 ratio. In this case, the larger subblock on the left, generated in the second split, has 576 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the quantization parameters are decoded for both the 576-pixel subblock on the left and the 192-pixel subblock on the right. Thus, in this case, three quantization parameters are decoded for the basic block, as shown in the center diagram of Figure 18(c).Furthermore, the four subblocks generated by the split are decoded in the order shown in the right-hand diagram of Figure 18(c). At this time, for each block unit that shares a quantization parameter, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order.

[0158] In the case of Figure 18(d), first, a flag of 1 is decoded to indicate that an asymmetric binary tree partition is being used. Next, a flag of 0 is decoded to indicate that it is a horizontal partition, and a flag of 0 is decoded to indicate that the partition ratio is 3:1. Then, as the first partition, an asymmetric binary tree partition is performed, dividing the base block into two subblocks horizontally in a ratio of 3:1. In this case, the larger subblock on the left has 768 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the larger subblock on the left and the smaller subblock on the right do not share quantization parameters. Subsequently, for the larger subblock on the left generated in the first partition, a flag of 1 is decoded to indicate that it is a vertical partition, and a flag of 1 is decoded to indicate that the partition ratio is 1:3. Then, as the second partition, an asymmetric binary tree partition is performed, dividing the larger subblock on the left generated in the first partition vertically into two subblocks in a ratio of 1:3. In this case, the larger subblock located at the bottom, generated in the second split, has 576 pixels, which is greater than or equal to the quantization control threshold of 512. Therefore, the quantization parameters are decoded for both the 192-pixel subblock at the top and the 576-pixel subblock at the bottom. Additionally, for the smaller subblock located on the right, generated in the first split, a flag of 1 is decoded to indicate that it is a vertical split, and a flag of 1 is decoded to indicate that the split ratio is 1:3. Then, as the second split, an asymmetric binary tree split is performed, dividing the smaller subblock located on the right, generated in the first split, into two subblocks vertically in a 1:3 ratio. In this case, the larger subblock located at the bottom, generated in the second split, has 192 pixels, which is less than the quantization control threshold of 512. Therefore, the 64-pixel subblock at the top and the 192-pixel subblock at the bottom share the quantization parameters, and one quantization parameter is decoded. Therefore, in this case, three quantization parameters are decoded for the basic block, as shown in the central diagram of Figure 18(d).Furthermore, the four subblocks generated by the partition are decoded in the order shown in the right-hand diagram of Figure 18(d). At this time, for each block unit that shares a quantization parameter, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order.

[0159] In the above explanation, we described a case where multiple asymmetric binary tree partitions are performed, resulting in subblock partitions that have the same shape as a single asymmetric quadtree partition. In this case, we described decoding a flag indicating the execution of a vertical or horizontal partition and a flag for specifying the partition ratio for both the first and second asymmetric binary tree partitions. However, this embodiment is not limited to this. In the second asymmetric binary tree partition, it is not necessary to decode the flag indicating the execution of a vertical or horizontal partition. For example, in the partitions of Figures 18(a) and 18(c), in the first asymmetric binary tree partition, 1 is decoded as the flag representing the execution of a vertical partition, and in the second asymmetric binary tree partition, the flag indicating the execution of a vertical or horizontal partition is not decoded. Then, by referring to the value of 1, which is the value of the flag indicating the execution of a vertical or horizontal partition used in the first asymmetric binary tree partition, a horizontal partition, which is the inverse value of 0, may be executed. Alternatively, in the partitions shown in Figures 18(a) and 18(c), the flag 1 is decoded in the first asymmetric binary tree partition to indicate the execution of a vertical partition. Of the two subblocks obtained in the first asymmetric binary tree partition, the upper subblock may also be decoded with a flag 1 indicating the execution of a vertical partition. Furthermore, the lower subblock does not decode a flag indicating the execution of a vertical or horizontal partition. The value of the flag indicating the execution of a vertical partition, which was decoded in the upper subblock, is 1, and a vertical partition may be executed, or the opposite value, 0, which signifies a horizontal partition, may be executed.

[0160] Furthermore, if both the upper and lower subblocks obtained in the first asymmetric binary tree partition, which divides vertically, are vertically partitioned, the base block will ultimately be divided vertically into four subblocks.

[0161] Alternatively, in Figures 18(b) and 18(d), in the first asymmetric binary tree split, 0 is decoded as the flag indicating the execution of a horizontal split, and in the second asymmetric binary tree split, the flag indicating the execution of a vertical or horizontal split is not decoded. Referencing the value of 0, which is the value of the flag indicating the execution of a vertical or horizontal split used in the first asymmetric binary tree split, a vertical split, which is the opposite value of 1, may be executed. Alternatively, also in Figures 18(b) and 18(d), in the first asymmetric binary tree split, 0 is decoded as the flag indicating the execution of a horizontal split. In the left-hand subblock of the two subblocks obtained in the first asymmetric binary tree split, 0 may be decoded as the flag indicating the execution of a horizontal split. Furthermore, in the right-hand subblock, the flag indicating the execution of a vertical or horizontal split is not decoded. Then, referencing the value of 0, which is the value of the flag indicating the execution of a horizontal split decoded in the left-hand subblock, a horizontal split may be executed, or a vertical split, which is the opposite value of 1, may be executed. Furthermore, if both the left and right subblocks obtained from the first asymmetric binary tree partition, which divides horizontally, are horizontally partitioned, the base block will be divided into four horizontal subblocks as a result. Also, in the second asymmetric binary tree partition, it is not necessary to decode the flag for identifying the partition ratio. For example, in the partitions of Figures 18(a) and 18(c), of the two subblocks obtained from the first asymmetric binary tree partition, the upper subblock is decoded with a flag of 0 to indicate that the partition ratio is 3:1. It is not necessary to decode the flag for identifying the partition ratio in the lower subblock. In this case, the partition ratio of the upper subblock may be used as the partition ratio of the lower subblock. Alternatively, in the partitions of Figures 18(b) and 18(d), of the two subblocks obtained from the first asymmetric binary tree partition, the left subblock is decoded with a flag of 1 to indicate that the partition ratio is 1:3. It is not necessary to decode the flag for identifying the partition ratio in the right subblock.In this case, the division ratio of the subblock located on the left may be used as the division ratio of the subblock located on the right. In this way, the flags indicating the execution of vertical or horizontal partitioning, or the flags for specifying the division ratio, which are necessary for the second asymmetric binary tree partitioning, are not decoded. As a result, a bitstream can be decoded that represents the flags indicating the execution of vertical or horizontal partitioning and the flags for specifying the division ratio with less coding than when decoding in the first and second asymmetric binary tree partitioning.

[0162] In this embodiment, a flag indicating the use of an asymmetric binary tree partition is decoded in the first block partition of the basic block. However, this embodiment is not limited to this. The flag indicating the use of an asymmetric binary tree partition may be decoded in the picture data portion before decoding each basic block. For example, one 1 may be decoded in the sequence header portion or picture header portion as a flag indicating the use of an asymmetric binary tree partition, and the flag indicating the use of an asymmetric binary tree partition may not be decoded in each basic block. The value decoded in the sequence header portion or picture header portion may then be used. In this way, the flag indicating the use of an asymmetric binary tree partition is not decoded in each basic block. By decoding one in the sequence header portion or picture header portion, a bitstream that represents the subblock partition by asymmetric partitioning with less coding can be decoded.

[0163] In the above description, a flag indicating the use of an asymmetric binary tree partition is decoded, followed by a flag indicating the execution of a vertical or horizontal partition. However, this embodiment is not limited to this. The flag indicating the use of an asymmetric binary tree partition may be decoded after the flag indicating the execution of a vertical or horizontal partition. For example, in the second partition after the basic block has been partitioned once using an asymmetric binary tree partition in the vertical direction, 1 may be decoded as the flag indicating the execution of a vertical partition. After that, 1 may be decoded as the flag indicating the use of an asymmetric binary tree partition. Alternatively, in the second partition of the two subblocks generated in the first partition, 0 may be decoded as the flag indicating the execution of a horizontal partition, followed by 0 as the flag indicating the use of a symmetric binary tree partition. In this way, in each subblock, the flag indicating the execution of a vertical or horizontal partition is decoded, followed by a flag indicating the use of an asymmetric binary tree partition. This allows selection of whether to use an asymmetric or symmetric partition in the partition of each subblock, and enables the decoding of a bitstream with finer control over the subblock partitioning.

[0164] Furthermore, Figure 19 illustrates the decoding of quantization parameters when multiple ternary tree partitioning or asymmetric binary tree partitioning are performed, resulting in subblock partitioning that produces the same partition shape as a single quadary tree partitioning. The thick lines and rectangles within the thick lines in Figure 19 are the same as in Figure 9, so no further explanation is provided. The numbers in Figure 19 indicate the number of pixels in each subblock. Figures 19(a) to (d) show the cases where the quantization control thresholds are 1024, 640, 640, and 256, respectively.

[0165] In Figure 19(a), the first partition is a ternary tree partition, dividing the data vertically into three subblocks in a ratio of 1:6:1. The central subblock, which has the largest number of pixels, has 768 pixels, which is less than the quantization control threshold of 1024. Subsequently, the second partition is an asymmetric binary tree partition, dividing the central subblock vertically in a ratio of 2:1. In this case, the larger subblock located at the top, generated by the second partition, has 512 pixels, which is less than the quantization control threshold of 1024. Therefore, in this case, as shown in the right-hand diagram of Figure 19(a), one quantization parameter common to the four subblocks is decoded. Alternatively, the first partition could be an asymmetric binary tree partition, dividing the data vertically in a ratio of 1:7. Subsequently, in the second partition, the larger subblock located at the bottom, generated by the first partition, could be binary-tree partitioned vertically in a ratio of 4:3. Furthermore, in the third partition, the smaller subblock located at the bottom, generated in the second partition, may be binary-tree partitioned vertically in a 2:1 ratio. Even in this case, one quantization parameter is decoded, as shown in the right-hand diagram of Figure 19(a). In the case of Figure 19(b), the binary-tree partition is asymmetric as shown in Figure 7(j), and is partitioned vertically in a 5:3 ratio. In this case, the larger subblock located at the top has 640 pixels, which is greater than or equal to the quantization control threshold of 640. Therefore, the two subblocks generated in the first partition do not share a quantization parameter. Subsequently, the larger subblock generated in the first partition is further binary-tree partitioned vertically in a 1:4 ratio. In this case, the larger subblock located at the bottom has 512 pixels, which is less than the quantization control threshold. Also, among the subblocks generated in the first partition, the smaller subblock located at the bottom is further binary-tree partitioned in a 2:1 ratio. In this case, the larger subblock located at the top has 256 pixels, which is less than the quantization control threshold. Therefore, in this case, two quantization parameters are decoded, as shown in the right-hand diagram of Figure 19(b).

[0166] In Figure 19(c), the first partition is a ternary tree partition, dividing the data into three subblocks in a vertical ratio of 1:6:1. In this case, the central subblock, which has the largest number of pixels, has 768 pixels, which is greater than or equal to the quantization control threshold of 640. Next, an asymmetric binary tree partition is performed, dividing the central subblock from the subblocks generated in the first partition in a vertical ratio of 2:1. In this case, the larger subblock located at the top has 512 pixels, which is less than the quantization control threshold. Therefore, the two subblocks generated in the second partition share the quantization parameters. Consequently, in this case, three quantization parameters are decoded for the base block, as shown in the right-hand diagram of Figure 19(c).

[0167] As shown in Figures 19(b) and 19(c), even when the basic block is similarly divided into four subblocks and the same quantization control threshold is used, it becomes possible to decode a different number of quantization parameters for the basic block. This is due to the difference in the combination of ternary tree partitioning and asymmetric binary tree partitioning. As a result, it is possible to decode a bitstream with finer control over the quantization parameters.

[0168] In Figure 19(d), the first partition is a ternary tree partition, dividing the data into three subblocks in a vertical ratio of 1:6:1. The central subblock, which has the largest number of pixels, has 768 pixels, which is greater than or equal to the quantization control threshold of 256. Subsequently, the second partition is an asymmetric binary tree partition, dividing the central subblock vertically in a ratio of 2:1. In this case, the larger upper subblock generated in the second partition has 512 pixels, which is greater than or equal to the quantization control threshold of 256. In this case, as shown in the right-hand diagram of Figure 19(d), the quantization parameters are decoded for each subblock. Therefore, four quantization parameters are decoded for the basic block. Alternatively, the first partition could be an asymmetric binary tree partition, dividing the data vertically in a ratio of 7:1. Subsequently, in the second partition, the larger upper subblock generated in the first partition could be binary-tree partitioned vertically in a ratio of 1:6. Furthermore, in the third partition, the larger subblock located at the bottom, generated in the second partition, may be split vertically in a binary tree with a 2:1 ratio. Even in this case, four quantization parameters are decoded, as shown in the right-hand diagram of Figure 19(d).

[0169] In the above explanation, we compared the subblock with the maximum number of pixels obtained in a single subblock division with the quantization control threshold, but this embodiment is not limited to this. The quantization parameters may be decoded based on the result of comparing the subblock with the minimum number of pixels obtained in a single subblock division with the quantization control threshold. The case of comparing with the subblock with the maximum number of pixels will be explained using Figure 20(a). In this case, the quantization control threshold is 192 pixels. The thick lines and the rectangles within the thick lines in Figure 20 are the same as in Figure 9, so we will omit the explanation. The numbers in Figure 20 indicate the number of pixels in each subblock. For clarity, in Figure 20, as in Figure 10, the quad division in Figure 8(b) is used for the first division of the recursive division, dividing the basic block into four 16x16 pixel blocks. Also, based on their position in the basic block, these are referred to here as the upper left region, upper right region, lower left region, and lower right region. In Figures 20(a) and (b), the upper left region is divided vertically using an asymmetric binary tree partition with a 5:3 ratio. The upper right region of the same figure is divided horizontally using an asymmetric binary tree partition with a 1:3 ratio. The lower left region of the same figure is divided horizontally using a ternary partition with a 1:2:1 ratio. The lower right region of the same figure is divided using an asymmetric quadtree partition, as shown in Figure 8(i), which generates four subblocks in a single partition.

[0170] In the upper left, lower left, and lower right regions of the left side of Figure 20(a), the subblocks with the largest number of pixels are 160, 128, and 144 pixels, respectively, and the number of pixels in each subblock is less than the quantization control threshold of 192. Therefore, one quantization parameter is decoded in the upper left, lower left, and lower right regions. On the other hand, in the upper right region, the subblock with the largest number of pixels is 192 pixels, which is above the quantization control threshold. Therefore, two quantization parameters are decoded in the upper right region, as shown in the right side of Figure 20(a). In other words, five quantization parameters are decoded in the entire right side of Figure 20(a).

[0171] The case of comparing with the subblock having the minimum number of pixels will be explained using Figure 20(b). In this case, the quantization control threshold is 96 pixels. The division state of the basic block in Figure 20(b) is the same as in Figure 20(a), so the explanation will be omitted. In the upper right, lower left, and lower right regions on the left side of Figure 20(b), the subblocks with the minimum number of pixels are 64 pixels, 64 pixels, and 16 pixels, respectively, and the number of pixels in all of these subblocks is less than the quantization control threshold of 96. Therefore, in the upper right, lower left, and lower right regions, one quantization parameter is decoded. On the other hand, in the upper left region, the subblock with the minimum number of pixels is 96, which is above the quantization control threshold. Therefore, in the upper left region, two quantization parameters are decoded, as shown in the figure on the right side of Figure 20(b). As a result, in the case of Figure 20(b), five quantization parameters are decoded. In this way, it becomes possible to share quantization parameters between different combinations of subblocks within a basic block divided into multiple subblocks. As a result, it becomes possible to decode bitstreams with more finely controlled quantization parameters.

[0172] Furthermore, although this embodiment describes the comparison as being made with the number of pixels, it may also be made with an index that substitutes for the number of pixels in a subblock. Specifically, when the number of pixels in a subblock is a power of 2 and the exponent of 2 is an integer, the quantization control threshold may be compared with the exponent. The comparison using the quantization control threshold will be explained using the table shown in Figure 21. However, the values ​​shown in Figure 21 are just examples. The first column of Figure 21 is the number of pixels in the subblocks generated by the division shape shown in Figure 8 or by recursive combinations thereof. The second column is the index corresponding to the exponent when the number of pixels in a subblock is expressed as a power of 2. There is a relationship between the number of pixels in a subblock and the index corresponding to the exponent, where the index decreases by 1 when the number of pixels in a subblock is halved. For example, the index corresponding to a number of pixels of 512 is 9, and the index corresponding to a number of pixels of 256 is 8. Furthermore, when the left side of Figure 9(e) is represented using powers of 2, the subblock pixel counts of 64, 128, and 256 correspond to indices 6, 7, and 8, respectively, and the quantization control threshold is 6. In this case, the quantization parameters are shared by the blocks on the right side of the figure, and the quantization parameters are decoded in association with the subblock containing the significance coefficient first in the decoding order for that unit. Thus, even when comparing the quantization control threshold with an index that substitutes for the number of pixels in a subblock, the quantization parameters can be controlled in the same way as when comparing the quantization control threshold with the number of pixels. Since the value representing the quantization control threshold is smaller than the index that substitutes for the number of pixels, it is possible to decode a bitstream in which the quantization control threshold is represented with less code, especially when encoding using Golomb coding.

[0173] Here, the process of decoding quantization parameters into each subblock based on the quantization control threshold will be explained using Figures 11 and 12. As mentioned above, in Figures 11(a) to (f) and Figures 12(a) to (d), the left side shows the type of block partition and the quantization parameter (Qp) used during encoding in each subblock. The left side of Figure 12(a) shows the four subblocks generated by partitioning the basic block once using the asymmetric quadtree partition shown in Figure 7(i). In contrast, the left side of Figure 12(b) shows that the basic block is first partitioned once using a binary tree partition that divides horizontally in a 3:1 ratio, generating two subblocks. Furthermore, the two subblocks generated in the first partition are partitioned vertically using a binary tree partition that divides vertically in a 1:3 ratio. As a result, the left side of Figure 12(b) shows the four subblocks generated by the repetition of the asymmetric binary tree partition. Also, the subblocks with diagonal lines indicate subblocks to which quantization parameters are associated. Furthermore, the thick-lined frame indicates the region where the quantization parameters, determined based on the quantization control threshold and the number of pixels in the processing target subblock, are shared. The method of comparison with the quantization control threshold is the same as that of the image encoding device in Embodiment 1. The center of the figure indicates whether each subblock has a significance coefficient. A significance coefficient is a non-zero coefficient among the residual coefficients. That is, "significance coefficient present" means that there is at least one non-zero residual coefficient within the subblock. The arrows shown on the right side of the figure indicate the decoding order. In the subblocks within the quantization parameter encoding unit, the quantization parameters are decoded in the subblock that first contains the significance coefficient in the decoding order. For example, in Figures 11(b) and 12(a), the first subblock to contain the significance coefficient in the decoding order is the upper right subblock, so the quantization parameters are decoded for that subblock. In this case, the lower left and lower right subblocks are within the region where the quantization parameters are shared, and the quantization parameters have already been decoded in the upper right subblock.In other words, the encoded data for the quantization parameters corresponding to the lower left and lower right subblocks does not exist in the bitstream, and the quantization parameters corresponding to the lower left and lower right subblocks are not decoded. In the quantization and dequantization processes for the lower left and lower right subblocks, QpA, which is the same quantization parameter as the upper right subblock, is used. Furthermore, since there is no significance coefficient in the upper left subblock, the dequantization process is not performed, but QpA, which is the same quantization parameter as the upper right subblock, is used for processes that use quantization parameters, such as the deblocking filter. Also, in Figure 11(f), the subblock that contains the significance coefficient first in the decoding order is the subblock located at the bottom, so the quantization parameters associated with that subblock are decoded. In Figure 11(f), the encoded data for the quantization parameters corresponding to the upper, center subblock does not exist in the bitstream, and the quantization parameters corresponding to the upper, center subblock are not decoded. However, in the upper and middle subblocks of Figure 11(f), similar to the upper left subblock of Figure 11(b), the same quantization parameter QpA as in the lower subblock is used for processes that utilize quantization parameters, such as deblocking filters. Also, in Figure 12(b), similar to Figure 12(a), the upper left subblock does not contain a significance coefficient, but the decoding order is different because the generation process of the four subblocks is different. In Figure 12(b), the first subblock to contain a significance coefficient in the decoding order is the lower left subblock, and the quantization parameter associated with that subblock is decoded. Since the upper left subblock of Figure 12(b) does not contain a significance coefficient, the quantization parameter is not decoded. On the other hand, the upper right and lower right subblocks of Figure 12(b) contain significance coefficients, but since the quantization parameter is encoded in the lower left subblock, the quantization parameter is not decoded in the upper right and lower right subblocks of Figure 12(b). Furthermore, in the quantization and dequantization processes in the upper right and lower right subblocks, the same quantization parameter QpA as in the lower left subblock is used.On the other hand, in the upper left subblock, since there is no significance coefficient, the inverse quantization process is not performed, similar to Figure 12(a). However, processes that use quantization parameters, such as deblocking filters, use the same quantization parameter QpA as in the lower left subblock. Also, in Figure 12(d), since there is no significance coefficient in the upper subblock, the quantization parameter is not decoded. On the other hand, since there is a significance coefficient in the lower subblock of Figure 12(d), the quantization parameter is decoded. In the upper subblock of Figure 12(d), similar to Figure 12(b), processes that use quantization parameters, such as deblocking filters, use the same quantization parameter QpA as in the lower subblock.

[0174] Thus, within the subblocks of the quantization parameter coding unit determined by the quantization control threshold, the quantization parameters are decoded for the subblock containing the significance coefficient first in the decoding order.

[0175] Furthermore, Figure 13 illustrates the case where there are subblocks in the region where quantization parameters are shared that do not contain any significance coefficients. The left side of Figure 13(a) shows a subblock partitioning similar to that in Figure 9, and the numbers in the figure indicate the number of pixels in each subblock. The left side of Figure 13(b) shows a subblock partitioning similar to that in Figure 10, and the numbers in the figure indicate the number of pixels in each subblock. The right side of Figure 13 is an example showing a shared region of quantization parameters determined based on the quantization control threshold and the number of pixels in each subblock, similar to Figures 9 and 10. Note that Figure 13(a) is an example where the quantization control threshold is 256, and Figure 13(b) is an example where it is 192. In Figure 13(a), we will explain the case where there are no significance coefficients in any of the three subblocks in the region corresponding to the quantization parameter QpD. In this case, the quantization parameter QpD for these three subblocks is not decoded. However, for processes such as deblocking filters, the same value as the quantization parameter decoded immediately before, i.e., QpC, is used. On the other hand, in Figure 13(b), it is assumed that there are no significance coefficients for both of the two subblocks in the region corresponding to the quantization parameter QpG. In this case, the quantization parameter QpG is not decoded in those two subblocks. In processes that use quantization parameters, such as deblocking filters, the QpF that was decoded immediately before is used. Although it is stated that in processes that use quantization parameters in quantization parameter coding units where no significance coefficients exist, the quantization parameter that was decoded immediately before in the decoding order is used, this embodiment is not limited to this. For example, in Figure 13(a), QpB, which is the quantization parameter of the quantization parameter coding unit adjacent to the top, may be used, or QpC, which is the quantization parameter adjacent to the left, may be used. In Figure 13(b), the quantization parameter of the quantization parameter coding unit adjacent to the top is QpC, and the quantization parameter adjacent to the left is QpF. Alternatively, a value calculated from the quantization parameters of multiple quantization parameter coding units, such as the mean value, may be used. Furthermore, initial values ​​for quantization parameters can be used for tiles or tile groups composed of multiple tiles.A tile is a unit that divides a frame, and it consists of at least one basic block.

[0176] Next, using Figure 14, we will explain how to decode the quantization parameters in a subblock that has been partitioned into a ternary tree, and then using Figure 16, we will explain how to decode the quantization parameters in a subblock that has been partitioned into an asymmetrical quadary tree. The meaning of the shaded subblocks and thick-lined frames in Figures 14 and 16 is the same as in Figure 11, so we will omit the explanation.

[0177] In Figure 14(a), all three subblocks contain significance coefficients. In this case, the quantization parameters are decoded in each of the three subblocks. That is, QpA is decoded in the first subblock, QpB in the second subblock, and QpC in the third subblock. In Figure 16(a), all four subblocks contain significance coefficients, and the quantization parameters are decoded in each of the subblocks. That is, QpA, QpB, QpC, and QpD are decoded in the order of decoding for the four subblocks.

[0178] In Figures 14(b) and 16(b), the first subblock in the decoding order does not contain the significance coefficient. In this case, the quantization parameter QpA is not decoded in the first subblock. On the other hand, the second and third subblocks in the decoding order contain the significance coefficient. Therefore, the quantization parameter QpB is decoded in the second subblock, and the quantization parameter QpC is decoded in the third subblock. Furthermore, in Figure 16(b), the quantization parameter QpD is decoded in the fourth subblock located in the lower right.

[0179] Here, we will explain, using Figure 15, how the quantization parameter is set for a subblock when the first subblock in the decoding order does not contain a significance coefficient. In Figure 15, the meaning of the thick border lines and diagonal lines is the same as in Figure 14. In Figure 15(a), the ternary tree-divided subblocks are the same as in Figure 14(b). That is, the first subblock in the decoding order does not contain a significance coefficient, while the second and third subblocks do. Furthermore, in Figure 15(b), the quadary tree-divided subblocks are the same as in Figure 16(b). That is, the first subblock in the decoding order does not contain a significance coefficient, while the second, third, and fourth subblocks do. As mentioned above, since the first subblock does not contain a significance coefficient, the quantization parameter QpA is not decoded. In this case, the quantization parameter of the subblock located to the left or above the subblock, or the subblock immediately preceding it in the decoding order, is set for that subblock. In Figures 15(a) and 15(b), Qpabove indicates the quantization parameter of the subblock located above, and Qpleft indicates the quantization parameter of the subblock located to the left. Furthermore, Qpprev indicates the quantization parameter of the immediately preceding subblock in the decoding order. Note that the squares labeled Qpabove, Qpleft, and Qpprev do not represent the shape of the subblocks, but rather indicate the quantization parameter of the immediately preceding subblock in the top, left, and decoding order, respectively.

[0180] In Figures 14(c) and 16(c), the second subblock in the decoding order does not contain the significance coefficient. In this case, the quantization parameter QpA is not decoded in the second subblock. At this time, the quantization parameter QpA of the first subblock, which is the preceding subblock in the decoding order, is set in the second subblock. On the other hand, in Figure 14(c), the first and third subblocks in the decoding order contain the significance coefficient. Therefore, the quantization parameter QpA is decoded in the first subblock, and the quantization parameter QpC is decoded in the third subblock. Also, in Figure 16(c), the first, third, and fourth subblocks in the decoding order contain the significance coefficient. Therefore, the quantization parameter QpA is decoded in the first subblock, QpB is decoded in the third subblock, and QpC is decoded in the fourth subblock.

[0181] In Figures 14(d) and 16(d), the first and second subblocks in the decoding order contain significance coefficients, but the third subblock does not. In this case, the quantization parameter QpB is not decoded in the third subblock. At this time, the quantization parameter QpB of the second subblock, which is the preceding subblock in the decoding order, is set in the third subblock. Furthermore, in Figure 16(d), the quantization parameter QpC is decoded in the fourth subblock.

[0182] In Figure 16(e), the first, second, and third subblocks in the decoding order contain significance coefficients, but the fourth subblock does not. In this case, the quantization parameter QpC is not decoded in the fourth subblock. At this time, the quantization parameter QpC of the third subblock, which is the immediately preceding subblock in the decoding order, is set in the fourth subblock.

[0183] Thus, as a result of comparing the quantization control threshold with the number of pixels in each subblock, when decoding quantization parameters in all multiple subblocks, the quantization parameters are associated with and decoded in relation to the subblock containing the significance coefficient. Furthermore, if a subblock does not contain a significance coefficient, the quantization parameters are not decoded in that subblock, but the quantization parameters of the immediately preceding subblock in the decoding order are set for that subblock. However, the quantization parameters set for subblocks without significance coefficients are not limited to these; they may also be the quantization parameters of adjacent subblocks to the left or above, or the average value of the quantization parameters of multiple subblocks.

[0184] In the inverse quantization / inverse transformation unit 204, inverse quantization is performed on the input residual coefficients to generate orthogonal transformation coefficients, and then an inverse orthogonal transformation is applied to reconstruct the prediction error. In the inverse quantization of each subblock, inverse quantization is performed using common quantization parameters for each region where quantization parameters are shared. The acquired prediction information is output to the image playback unit 205.

[0185] The image playback unit 205 appropriately references the frame memory 206 based on the prediction information input from the decoding unit 203 and plays back the predicted image. Image data is then played back from this predicted image and the prediction error input from the inverse quantization / inverse transform unit 204, input into the frame memory 206, and stored. The stored image data is used as a reference during prediction.

[0186] In the in-loop filter unit 207, similar to 109 in Figure 1, the playback image is read from the frame memory 206 and in-loop filtering is performed, such as deblocking filtering and sample adaptive offsetting. The filtered image is then input back into the frame memory 206.

[0187] The playback image stored in the frame memory 206 is ultimately output externally from terminal 208.

[0188] Figure 4 is a flowchart showing the image decoding process in an image decoding device.

[0189] First, in step S401, the separation and decoding unit 202 separates the bitstream into coded data related to the decoding process and coefficients, decodes the coded data of the header portion, and obtains quantization control threshold information.

[0190] In this embodiment, the number of pixels in a subblock is decoded as information regarding the quantization control threshold. The number of pixels in a subblock is a positive integer; for example, in the partitioning example shown in Figure 9(b) or Figure 10(a), 512 is decoded as the quantization control threshold. However, in this embodiment, the information regarding the quantization control threshold is not limited to the number of pixels in a subblock. An index calculated based on the number of pixels in a subblock may also be decoded. Here, the numerical value to be decoded as the index will be explained using the table shown in Figure 21. However, the numerical value shown in Figure 21 is just an example. In this embodiment, when the number of pixels in a subblock can be expressed as a power of 2, the value of the exponent is decoded as the index. For example, when the number of pixels in a subblock is 512, the corresponding quantization control threshold is 512 = 2^9, so the numerical value representing the exponent of 2 shown in the second column of Figure 21, 9, is decoded. Also, when the number of pixels in a subblock is 128, 128 = 2^7, so 7 is decoded, and so on. On the other hand, if the ratio of each subblock to the number of pixels in the base block can be calculated based on the number of pixels in the base block, and the reciprocal of that value can be expressed as a power of 2, then the value of that power of 2 may be used as the index for decoding. For example, if the base block has 1024 (32 × 32) pixels and the quantization control threshold is 1024, then the ratio of the number of pixels in the base block to the number of pixels is 1, and its reciprocal is also 1, so it can be expressed as 1 = 2^0. In this case, 0 is decoded, as shown in the 5th column of Figure 21. Alternatively, if the quantization control threshold is 64, then the ratio of the number of pixels in the base block to the number of pixels is 1 / 16, and its reciprocal is 16, so it can be expressed as 16 = 2^4. In this case, 4 is decoded, and so on.

[0191] This allows decoding 0 when the quantization control threshold is at its maximum value (the same as the number of pixels in the basic block). In particular, when decoding a bitstream encoded using Golomb coding, it is possible to decode a bitstream with the quantization control threshold encoded using fewer bits.

[0192] Furthermore, if the number of pixels in a subblock is expressed in combination with a power of 2 and a power of 3, it is also possible to decode two indices corresponding to both the power of 2 and the power of 3. For example, if the number of pixels in a subblock is 192, then 192 = 2^6 × 3^1, so we decode two indices: the power of 2 m=6 shown in the second column of Figure 21 and the power of 3 n=1 shown in the third column. Also, if the number of pixels in a subblock is 80, then 80 = 2^6 × 5^1, so we decode two indices: the power of 2 m=6 and the power of 5 k=1. Moreover, if the number of pixels in a subblock is 120, then 120 = 2^3 × 3^1 × 5^1. Therefore, we decode three indices: the power of 2 m=3 shown in the second column of Figure 21, the power of 3 n=1 shown in the third column of the same figure, and the power of 5 k=1 shown in the fourth column of the same figure. This makes it possible to set quantization control thresholds that cannot be expressed as powers of 2, enabling the decoding of bitstreams with more finely controlled quantization parameters.

[0193] Furthermore, in this embodiment, while it is stated that a plurality of corresponding indices are decoded for the number of pixels in a subblock and the exponent used to represent the quantization control threshold, the embodiment is not limited to this. For example, the index of the exponent m of 2 may be decoded as a required value, and flags representing whether or not each of the other exponents are encoded may be decoded. For example, if the quantization control threshold is calculated by multiplying a power of 2 and a power of 3, such as 192, the flag representing the encoding of the exponent of 3 and the value of the exponent of 3 may be decoded. That is, in the case of 192, 1 may be decoded as the flag representing the encoding of the exponent of 3, 0 as the flag representing the non-encoding of the exponent of 5, 6 as the exponent of 2, and 1 as the exponent of 3, as shown in Figure 21. On the other hand, if the quantization control threshold can be calculated by multiplying a power of 2 and a power of 5, such as 80, the flag representing the encoding of the exponent of 5 and the value of the exponent of 5 may be decoded. In other words, in the case of 80, we may decode 0 as the flag representing the unencoded power of 3, 1 as the flag representing the encoded power of 5, 4 as the power of 2, and 1 as the power of 5. Furthermore, if the quantization control threshold can be calculated by multiplying the values ​​of powers of 2, 3, and 5, such as 120, we may decode the flags representing the encoded powers of 2, 3, and 5, and the values ​​of the powers of 2, 3, and 5. In other words, we may decode 1 as the flag representing the encoded power of 3, 1 as the flag representing the encoded power of 5, 3 as the power of 2, 1 as the power of 3, and 1 as the power of 5. In this way, in addition to the index of the power of 2, only the indices necessary when the flag is true can be decoded.

[0194] In this embodiment, we decode flags indicating the existence of each exponent for prime numbers greater than or equal to 3 that constitute the quantization control threshold, but we are not limited to this. For example, we may decode information representing the value of the largest prime number that constitutes the value of the quantization control threshold. That is, if the largest prime number is 5, counting prime numbers from smallest to largest, 5 is the third, so we may decode index 2 as information representing the value of the largest prime number that constitutes the value of the quantization control threshold. Also, if there are two prime numbers other than 2 that constitute the quantization control threshold, namely 3 and 11, the largest prime number is 11, and 11 is the fifth prime number from smallest to largest, so we may decode index 4. In this way, the index is uniquely determined by counting prime numbers from smallest to largest. Furthermore, if the index is 0, it may mean that the exponent corresponding to prime numbers greater than or equal to 3 is not used in the quantization control threshold. In this case, the value of the exponent corresponding to prime numbers greater than or equal to 3 is 0. If the largest prime number is 5, in addition to index 2, we may decode the values ​​of the exponents of 2, 3, and 5. For example, if the quantization control threshold is 720, then 4, 2, and 1 may be decoded, as shown in Figure 21. Similarly, if the quantization control threshold is 640, then 7, 0, and 1 may be decoded. In this way, by using information representing the value of the largest prime number that constitutes the quantization control threshold, a bitstream encoded with an arbitrary number of pixels using several power exponents can be efficiently decoded.

[0195] Note that the numerical values ​​corresponding to powers of 2 are not limited to exponents of 2. For example, they may be derived from the reciprocal of the ratio of the number of pixels in the basic block. That is, if the size of the basic block is 1024 (32 × 32) pixels, the exponent of 2 corresponding to 64 pixels is 6, but the ratio to 1024 pixels is 1 / 16, and as shown in the 5th column of Figure 21, its reciprocal is 16 = 2^4. Therefore, when the quantization control threshold is 192, instead of decoding 6 as the numerical value corresponding to a power of 2, we may decode 4, and then decode flag 1 indicating the encoding of an exponent of 3, and the exponent of 3, 1. In this way, by using a value derived from the reciprocal of the ratio of the number of pixels in the basic block, it is possible to decode a bitstream that represents the quantization control threshold with fewer bits than encoding the exponent.

[0196] In this embodiment, an index based on the value of a power exponent is decoded as the quantization control threshold, but this is not limited to this. Indexes assigned sequentially from the maximum number of pixels to the minimum number of pixels in a subblock may also be decoded. For example, as shown in Figure 21, if there are 19 different values ​​for the number of pixels in a subblock, with the maximum value being 1024 pixels and the minimum value being 8 pixels, index 0 may be assigned to 1024 pixels and index 18 to 8 pixels. In this case, if the quantization control threshold is 1024, 0 may be decoded, and if it is 720, 2 may be decoded. By decoding 0 when the quantization control threshold is at its maximum value (the same as the number of pixels in the base block), it is possible to decode a bitstream that encodes the quantization control threshold with fewer bits, especially when encoding using Golomb or similar methods.

[0197] In step S402, the decoding unit 203 decodes the coded data separated in step S401 and obtains block partitioning information, residual coefficients, prediction information, and quantization parameters.

[0198] In step S403, the inverse quantization / inverse transformation unit 204 performs inverse quantization on the residual coefficients in sub-block units and then performs an inverse orthogonal transformation to obtain the prediction error. Specifically, it compares the quantization control threshold information with the size determined based on the number of pixels in the sub-blocks determined from the acquired block division information. As a result, the regions (sub-blocks) in which the quantization parameters are shared are determined. Then, the inverse quantization process is performed based on the quantization parameters assigned to each sub-block.

[0199] In step S404, the image playback unit 205 plays back a predicted image based on the prediction information acquired in step S402. Furthermore, it plays back image data from the played-back predicted image and the prediction error generated in step S403.

[0200] In step S405, the control unit 200 of the image decoding device determines whether decoding of all blocks in the frame has been completed. If it has been completed, it proceeds to step S406; otherwise, it returns to step S402 and targets the next block.

[0201] In step S406, the in-loop filter unit 207 performs in-loop filtering on the image data reproduced in step S404, generates a filtered image, and terminates the process.

[0202] With the above configuration and operation, it is possible to decode a bitstream with reduced data volume by decoding the quantization parameters using quantization control threshold information.

[0203] In the image decoding device, as shown in Figure 6(a), the bitstream containing the quantization control threshold information in the picture header is decoded. However, the encoding location of the information is not limited to this. As shown in Figure 6(b), it may be encoded in the sequence header of the image, or at any other location.

[0204] In this embodiment, it was explained that the number of pixels in each subblock is compared with one quantization control threshold to determine whether or not quantization parameters should be shared among multiple subblocks. However, this embodiment is not limited to this. Two or more quantization control thresholds may be used. Here, the case in which quantization parameters are decoded using two quantization control thresholds will be explained using Figure 10(e).

[0205] In the cases of Figures 10(c) and 10(d) described above, the quantization control threshold is compared with the subblock with the largest number of pixels among the multiple subblocks generated by performing an asymmetric partition once. The quantization control threshold compared with the subblock with the largest number of pixels in this way is called the first quantization control threshold (hereinafter referred to as the first threshold). In contrast, the quantization control threshold compared with the subblock with the smallest number of pixels among the multiple subblocks generated by performing an asymmetric partition once is called the second quantization control threshold (hereinafter referred to as the second threshold). In the case of Figure 10(e), the first threshold is 128 and the second threshold is 32. When using the first and second thresholds, the upper left region on the left side of Figure 10(e) is divided into four subblocks by a single asymmetric quadtree partition, and the lower left subblock with the largest number of pixels has 144 pixels, which is greater than or equal to the first threshold. However, at the same time, the upper right subblock with the smallest number of pixels has 16 pixels, which is less than the second threshold. Therefore, since the comparison conditions for the first and second thresholds are not simultaneously met, the four subblocks share one quantization parameter, as shown on the right side of Figure 10(e). Next, in the upper right and lower left regions of the left side of Figure 10(e), the area is divided into two subblocks by an asymmetric binary tree partition. Of these, the larger subblock located on the upper side in the upper right region, and the larger subblock located on the left side in the lower left region, have 192 pixels, which is greater than or equal to the first threshold of 128. At the same time, the smaller subblock located on the lower side in the upper right region, and the smaller subblock located on the right side in the lower left region, have 64 pixels, which is greater than or equal to the second threshold of 32. Therefore, in this case, the quantization parameter is decoded in each of the two subblocks. Also, in the lower right region, as in Figure 10(c) above, the asymmetric binary tree partition is performed twice. In the first binary tree partition, the larger subblock located on the right side has 192 pixels, which is greater than or equal to the first threshold. At the same time, the smaller subblock located on the left has 64 pixels, which is greater than or equal to the second threshold of 32.Furthermore, in the second binary tree partition targeting the larger subblock located on the right side, generated in the first partition, the larger subblock located on the upper side has 144 pixels, which is above the first threshold. At the same time, the smaller subblock located on the lower side has 48 pixels, which is above the second threshold. Therefore, in the lower right region of Figure 10(e), quantization parameters are decoded for each of the three subblocks. In this way, by using two quantization control thresholds, the quantization parameters are shared within the block units on the right side of Figure 10(e). Furthermore, within the unit of that block, one quantization parameter is decoded in association with the subblock containing the significance coefficient at the beginning of the decoding order. In this case, eight quantization parameters are decoded, as shown on the right side of Figure 10(e). In contrast, Figure 10(d) uses only the first threshold for comparison, and the threshold value is 128, the same as in Figure 10(e). In this case, eleven quantization parameters are decoded, as shown on the right side of Figure 10(d). Therefore, in the case of Figure 10(e), which uses two quantization control thresholds, three fewer quantization parameters are decoded compared to the case of Figure 10(d), which uses one quantization control threshold. In this way, by using a second threshold in addition to the first threshold, the number of quantization parameters to be decoded can be set more flexibly compared to the case where only one quantization control threshold is used, and a bitstream with suppressed over-encoding of quantization parameters can be decoded.

[0206] It was explained that quantization parameters are decoded in each subblock generated by the asymmetric partitioning only when both the comparison using the first quantization control threshold and the comparison using the second quantization control threshold are valid. However, this embodiment is not limited to this. For example, quantization parameters may be decoded in each subblock when at least one comparison is valid. A specific example is illustrated using Figure 10(c). In Figure 10(c), the first threshold is 192 and the second threshold is 32. In this case, in the upper left region on the left side of Figure 10(c), the largest subblock, 144 pixels, is less than the first threshold, and the smallest subblock, 16 pixels, is less than the second threshold. Therefore, since no comparison is valid, one quantization parameter may be decoded in the upper left region, as shown on the right side of Figure 10(c). In the upper right and lower left regions of Figure 10(c), the largest subblock, 192 pixels, is greater than or equal to the first threshold, and the smallest subblock, 64 pixels, is greater than or equal to the second threshold. Therefore, since one or more comparisons are valid, two quantization parameters may be decoded in the upper right and lower left regions, as shown on the right side of Figure 10(c). In the lower right region on the left side of Figure 10(c), two asymmetric binary tree partitions are performed. In the first binary tree partition, the larger subblock on the right has 192 pixels, which is greater than or equal to the first threshold, and the smaller subblock on the left has 64 pixels, which is greater than or equal to the second threshold. Furthermore, in the second binary tree partition targeting the larger subblock on the right generated in the first partition, the larger subblock on the upper side has 144 pixels, which is less than the first threshold of 192. Also, the smaller subblock on the lower side has 48 pixels, which is greater than or equal to the second threshold of 32. Therefore, since a comparison using the second threshold is valid, three quantization parameters may be decoded in the lower right region on the left side of Figure 10(c), as shown in the lower right region on the right side of Figure 10(d). Therefore, if only the first threshold is used, seven quantization parameters are decoded, and if at least one of the first and second thresholds is satisfied, eight quantization parameters may be decoded.

[0207] Thus, if at least one comparison is successful, quantization parameters can be decoded in more subblocks, enabling the decoding of a bitstream with fine-grained image quality adjustments at the subblock level.

[0208] Here, the decoding of the two quantization control thresholds will be explained. In this embodiment, the number of pixels in the subblock is decoded as information regarding the quantization control thresholds. The number of pixels in the subblock is a positive integer. For example, in the partitioning example shown in Figure 10(e), 128 is decoded as the first threshold, and 32 is decoded as the second threshold. The decoding of the second threshold may be a ratio with respect to the first threshold. For example, if the first and second thresholds are 128 and 32, the configuration may be to decode 128 and 4. Furthermore, the decoding of the second threshold may be the difference between it and the first threshold. For example, if the first and second thresholds are 192 and 128, 192 and 64 can also be decoded.

[0209] In this embodiment, the information regarding the quantization control threshold is not limited to the number of pixels in the subblock. An index calculated based on the number of pixels in the subblock may be decoded. Here, the numerical values ​​to be decoded as the index will be explained using the table shown in Figure 21. However, the numerical values ​​shown in Figure 21 are just examples. In this embodiment, if the number of pixels in the subblock can be expressed as a power of 2, the value of the exponent may be decoded as the index. For example, if the number of pixels in the subblock is 128, the first threshold corresponding to this is 128 = 2^7, so the numerical value representing the exponent of 2 shown in the second column of Figure 21, which is 7, may be decoded. Furthermore, if the number of pixels in the subblock is 32, the second threshold corresponding to this is 32 = 2^5, so 5 may be decoded. Also, if the second threshold is expressed as the difference between it and the first threshold, 2 may be decoded. Alternatively, if the number of pixels in the subblock is 256, 8 may be decoded as the first threshold, and if the number of pixels in the subblock is 16, 4 may be decoded as the second threshold. In this case, since the first threshold is twice the second threshold, we can decode 2 instead of 8.

[0210] Furthermore, if the number of pixels in a subblock is expressed in combination with a power of 2 and a power of 3, two indices corresponding to both the power of 2 and the power of 3 may be decoded for the first and second thresholds, respectively. For example, if the number of pixels corresponding to the first threshold is 144, then 144 = 2^4 × 3^2, so two indices may be decoded: the power of 2 m=4 shown in the second column of Figure 21 and the power of 3 n=2 shown in the third column. Additionally, if the number of pixels corresponding to the second threshold is 10, then 10 = 2^1 × 5^1, so the power of 2 m=1 and the power of 5 k=1 are decoded as two indices. That is, 6 and 1 are decoded as the first threshold, and 2 and 5 are decoded as the second threshold, so a total of four indices may be decoded.

[0211] Furthermore, in this embodiment, image data is input frame by frame, and the bitstream generated by encoding is decoded. However, the target of the decoding process is not limited to the bitstream from which the image data has been encoded. For example, feature data used in machine learning, such as object recognition, may be input in a two-dimensional shape, and the bitstream generated by encoding is decoded. This makes it possible to efficiently decode a bitstream from which feature data used in machine learning has been encoded.

[0212] <Embodiment 3> In the above embodiment, each processing unit shown in Figures 1 and 2 was described as being composed of hardware. However, the processing performed by each processing unit shown in these figures may also be composed of computer programs.

[0213] Figure 5 is a block diagram showing an example of a computer hardware configuration applicable to the image display device according to each of the above embodiments.

[0214] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and ROM 503, and also executes the processes described above, which are performed by the image processing apparatus according to each embodiment. In other words, the CPU 501 functions as the processing unit shown in Figures 1 and 2.

[0215] RAM 502 has an area for temporarily storing computer programs and data loaded from the external storage device 506, and data acquired from the outside via the I / F (interface) 507. Furthermore, RAM 502 has a work area used by the CPU 501 when executing various processes. That is, RAM 502 can be allocated as, for example, frame memory, or various other areas can be provided as appropriate.

[0216] ROM 503 stores the computer's configuration data and boot program. The operation unit 504 consists of a keyboard and mouse, allowing the computer's user to input various instructions to the CPU 501. The display unit 505 displays the processing results from the CPU 501. The display unit 505 is, for example, a liquid crystal display.

[0217] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores the OS (operating system) and computer programs that enable the CPU 501 to implement the functions of each part shown in Figures 1 and 2. Furthermore, the external storage device 506 may also store the image data to be processed.

[0218] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate, according to the control of the CPU 501, and become subject to processing by the CPU 501. Other devices such as LANs, the Internet, projection devices, and display devices can be connected to the I / F 507, and this computer can acquire and transmit various information via this I / F 507. 508 is a bus that connects the above-mentioned parts.

[0219] The operation resulting from the above configuration is controlled primarily by the CPU 501, as described in the flowchart above.

[0220] The objective of this embodiment can also be achieved by supplying a storage medium containing the code of a computer program that realizes the aforementioned functions to a system, and by the system reading and executing the computer program code. In this case, the computer program code read from the storage medium itself realizes the functions of the embodiment described above, and the storage medium containing the computer program code constitutes this embodiment. Furthermore, this also includes cases in which an operating system (OS) or the like running on the computer performs some or all of the actual processing based on the instructions in the program code, and the aforementioned functions are realized through that processing.

[0221] Furthermore, it may also be implemented in the following form: that is, computer program code read from a storage medium is written to the memory of a function expansion card inserted into a computer or a function expansion unit connected to a computer. Then, based on the instructions in that computer program code, the CPU or other components of the function expansion card or function expansion unit perform some or all of the actual processing to realize the aforementioned function.

[0222] When this embodiment is applied to the storage medium described above, the storage medium will store the code of a computer program corresponding to the flowchart described earlier.

[0223] This embodiment is used in encoding and decoding devices that encode and decode still images and videos. In particular, it can be applied to encoding and decoding methods that use quantization processing.

[0224] The following disclosures of embodiments include the following sections:

[0225] (Section 1) A method for decoding an image, A process of decoding an image which is divided into one or more blocks and further divided into one or more subblocks from the said blocks, The division from the block to the subblock includes an asymmetric division, The process involves decoding the quantization parameters in the asymmetrically divided block based at least on information regarding the number of pixels in the subblock, A decoding method characterized by having the following features.

[0226] (Section 2) The decoding method according to item 1, characterized by comparing the number of pixels of the subblock with a threshold and decoding the quantization parameters according to the result of the comparison.

[0227] (Section 3) The decoding method according to item 2, characterized in that, in the asymmetrically divided blocks, if the number of pixels in the subblocks is greater than or equal to the threshold, the quantization parameters are decoded for each subblock.

[0228] (Section 4) The decoding method according to any one of claims 1 to 3, characterized in that the asymmetric partition includes a two-part partition that divides the block to be partitioned into two.

[0229] (Section 5) The decoding method according to any one of claims 1 to 3, characterized in that the asymmetric partitioning includes a four-part partitioning which divides the block to be partitioned into four parts.

[0230] (Section 6) The decoding method according to claim 4, characterized in that the subblock generated by the two divisions of the asymmetrical division can be further divided into two subblocks by the two divisions of the asymmetrical division.

[0231] (Section 7) If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is equal to or greater than the threshold, the quantization parameter is decoded for each subblock included in the asymmetrically divided block. The decoding method according to item 3, characterized by the features described herein.

[0232] (Section 8) The threshold is a first threshold, If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to the first threshold, and the subblock with the smallest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to a second threshold that is smaller than the first threshold, then the quantization parameters are decoded for each subblock included in the asymmetrically divided block. The decoding method according to item 7, characterized in that

[0233] (Section 9) If all of the subblocks included in the asymmetrically divided block are below the threshold, the subblocks are decoded so that one quantization parameter is shared among them. The decoding method according to item 3, characterized by the features described herein.

[0234] (Section 10) A method for encoding an image, A process of encoding an image which is divided into one or more blocks, and which is further composed of one or more subblocks divided from the said blocks, The division from the block to the subblock includes an asymmetric division, The steps include: encoding quantization parameters in the asymmetrically divided block based at least on information regarding the number of pixels in the subblock; An encoding method characterized by having the following:

[0235] (Section 11) The encoding method according to item 10, characterized by comparing the number of pixels of the subblock with a threshold and encoding the quantization parameters according to the result of the comparison.

[0236] (Section 12) The encoding method according to item 11, characterized in that, in the asymmetrically divided blocks, if the number of pixels in the subblocks is greater than or equal to the threshold, a quantization parameter is encoded for each subblock.

[0237] (Section 13) The encoding method according to any one of claims 10 to 12, characterized in that the asymmetric partitioning includes a two-part partitioning that divides the block to be partitioned into two.

[0238] (Section 14) The encoding method according to any one of claims 10 to 13, characterized in that the asymmetric partitioning includes a four-part partitioning which divides the block to be partitioned into four parts.

[0239] (Section 15) The encoding method according to claim 13, characterized in that the subblock generated by the two divisions of the asymmetrical division can be further divided into two subblocks by the two divisions of the asymmetrical division.

[0240] (Section 16) If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is equal to or greater than the threshold, then a quantization parameter is encoded for each subblock included in the asymmetrically divided block. The encoding method according to item 12, characterized by the features described herein.

[0241] (Section 17) The threshold is a first threshold, If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to the threshold, and the subblock with the smallest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to a second threshold that is smaller than the first threshold, then a quantization parameter is encoded for each subblock included in the asymmetrically divided block. The encoding method according to item 16, characterized in that

[0242] (Section 18) If all of the subblocks included in the asymmetrically divided block are below the threshold, the subblocks are encoded so that one quantization parameter is shared among them. The encoding method according to item 12, characterized by the features described herein.

[0243] (Section 19) A decoding device for decoding images, Means for decoding an image that is divided into one or more blocks, and further composed of one or more subblocks divided from the said blocks, The division from the block to the subblock includes an asymmetric division, In the asymmetrically divided block, means for decoding quantization parameters based at least on information regarding the number of pixels of the subblock, A decoding device characterized by having the following features.

[0244] (Section 20) An encoding device for encoding images, Means for encoding an image that is divided into one or more blocks, and further composed of one or more subblocks divided from the said blocks, The division from the block to the subblock includes an asymmetric division, In the asymmetrically divided block, means for encoding quantization parameters based at least on information regarding the number of pixels in the subblock, An encoding device characterized by having the following features.

[0245] (Section 21) A program that, when read and executed by a computer, causes the computer to perform each step of the decoding method described in any one of items 1 to 9.

[0246] (Section 22) A program that, when read and executed by a computer, causes the computer to perform each step of the encoding method described in any one of items 10 to 18. [Explanation of symbols]

[0247] 100, 200 Control Unit Terminals 101, 112, 201, 208 102 Block division section 103 Generation part 104 Prediction Section 105 Conversion / Quantization Section 106, 204 Inverse Quantization / Inverse Transform Section 107, 205 Image playback section 108,206 frame memory 109, 207 In-loop filter section 110 Encoding section 111 Integrated Encoding Unit 202 Separation and Decoding Unit 203 Decoding Unit

Claims

1. A method for decoding an image, A process of decoding an image that is divided into one or more blocks, and further divided into one or more sub-blocks from the said blocks, The division from the block to the subblock includes an asymmetric division, The process involves decoding the quantization parameters in the asymmetrically divided block based at least on information regarding the number of pixels in the subblock, A decoding method characterized by having the following features.

2. The decoding method according to claim 1, characterized in that the number of pixels of the subblock and a threshold value are compared, and the quantization parameters are decoded according to the result of the comparison.

3. The decoding method according to claim 2, characterized in that, in the asymmetrically divided blocks, if the number of pixels in the subblocks is greater than or equal to the threshold, the quantization parameters are decoded for each subblock.

4. The decoding method according to claim 1, characterized in that the asymmetric partition includes a two-part partition that divides the block to be partitioned into two.

5. The decoding method according to claim 1, characterized in that the asymmetric partitioning includes a four-part partitioning which divides the block to be partitioned into four parts.

6. The decoding method according to claim 4, characterized in that the subblock generated by the two divisions of the asymmetrical division can be further divided into two subblocks by the two divisions of the asymmetrical division.

7. If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is equal to or greater than the threshold, the quantization parameter is decoded for each subblock included in the asymmetrically divided block. The decoding method according to feature 3.

8. The threshold is a first threshold, If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to the first threshold, and the subblock with the smallest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to a second threshold that is smaller than the first threshold, then the quantization parameters are decoded for each subblock included in the asymmetrically divided block. The decryption method according to feature 7.

9. If all of the subblocks included in the asymmetrically divided block are below the threshold, the decodes so that one quantization parameter is shared among the subblocks. The decoding method according to feature 3.

10. A method for encoding an image, A process of encoding an image which is divided into one or more blocks, and which is further composed of one or more subblocks divided from the said blocks, The division from the block to the subblock includes an asymmetric division, The steps include: encoding quantization parameters in the asymmetrically divided block based at least on information regarding the number of pixels in the subblock; An encoding method characterized by having the following:

11. The encoding method according to claim 10, characterized in that the number of pixels of the subblock and a threshold are compared, and the quantization parameters are encoded according to the result of the comparison.

12. The encoding method according to claim 11, characterized in that, in the asymmetrically divided blocks, if the number of pixels in the subblocks is greater than or equal to the threshold, a quantization parameter is encoded for each subblock.

13. The encoding method according to claim 10, characterized in that the asymmetric partitioning includes a two-part partitioning that divides the block to be partitioned into two.

14. The encoding method according to claim 10, characterized in that the asymmetric partitioning includes a four-part partitioning which divides the block to be partitioned into four parts.

15. The encoding method according to claim 13, characterized in that the subblock generated by the two divisions of the asymmetric division can be further divided into two subblocks by the two divisions of the asymmetric division.

16. If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is equal to or greater than the threshold, then a quantization parameter is encoded for each subblock included in the asymmetrically divided block. The encoding method according to feature 12.

17. The threshold is a first threshold, If the subblock with the largest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to the threshold, and the subblock with the smallest number of pixels among the multiple subblocks included in the asymmetrically divided block is greater than or equal to a second threshold that is smaller than the first threshold, then a quantization parameter is encoded for each subblock included in the asymmetrically divided block. The encoding method according to claim 16.

18. If all of the subblocks included in the asymmetrically divided block are below the threshold, the subblocks are encoded so that one quantization parameter is shared among them. The encoding method according to feature 12.

19. A decoding device for decoding images, Means for decoding an image that is divided into one or more blocks, and further divided into one or more subblocks from the said blocks, The division from the block to the subblock includes an asymmetric division, In the asymmetrically divided block, means for decoding quantization parameters based at least on information regarding the number of pixels of the subblock, A decoding device characterized by having the following features.

20. An encoding device for encoding images, Means for encoding an image which is divided into one or more blocks and further composed of one or more subblocks divided from the blocks, The division from the block to the subblock includes an asymmetric division, In the asymmetrically divided block, means for encoding quantization parameters based at least on information regarding the number of pixels in the subblock, An encoding device characterized by having the following features.

21. A program that, when read and executed by a computer, causes the computer to perform each step of the decoding method described in any one of claims 1 to 9.

22. A program that, when read and executed by a computer, causes the computer to perform each step of the encoding method described in any one of claims 10 to 18.

Images