Image encoding device, image encoding method, image decoding device, image decoding method

Abstract

To provide a technique to generate a quantization matrix corresponding to rectangular orthogonal transformation.SOLUTION: All elements on the first row, the third row, the fifth row, and the seventh row of a first quantization matrix are removed without increasing elements of the first quantization matrix of 8×8 size in a horizontal direction, thereby reducing the elements of the first quantization matrix of 8×8 size in the horizontal direction. Elements on the m-th column and the 2n-th row (m, n are an integer of 0 or more) in the first quantization matrix are arranged as elements on the 2m-th column and the n-th row and elements on the (2m+1)-th column and the n-th row of a second quantization matrix to increase in a vertical direction, the elements in the first quantization matrix of 8×8 size, thereby generating the second quantization matrix. A third quantization matrix of 8×16 size can be generated from the first quantization matrix.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to image encoding and decoding techniques. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter referred to as HEVC) is known as a coding method for compressing and recording moving images. To improve coding efficiency, HEVC uses basic blocks larger than conventional macroblocks (16 pixels x 16 pixels). These large basic blocks are called coding tree units (CTUs) and can be up to 64 pixels x 64 pixels in size. CTUs are further divided into sub-blocks, which serve as units for prediction and transformation.

[0003] Furthermore, HEVC uses a matrix called a quantization matrix, which weights coefficients after orthogonal transformation (hereinafter referred to as orthogonal transform coefficients) according to frequency components. By further reducing data of high-frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technology for encoding such a quantization matrix.

[0004] In recent years, efforts have been initiated to internationally standardize a more efficient coding method as a successor to HEVC. The Joint Video Experts Team (JVET) was established between ISO / IEC and ITU-T, and standardization is underway as the Versatile Video Coding (VVC) coding method (hereafter referred to as VVC). To improve coding efficiency, rectangular sub-block-based intra-prediction and orthogonal transform methods are being considered in addition to the conventional square sub-block-based intra-prediction and orthogonal transform methods. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-38758 Summary of the Invention [Problem to be solved by the invention]

[0006] As with HEVC, the introduction of quantization matrices is being considered for VVC. Furthermore, not only square but also rectangular subblock divisions and corresponding orthogonal transform shapes are being considered for VVC. Because the distribution of each orthogonal transform coefficient differs depending on the orthogonal transform shape, it is desirable to apply an optimal quantization matrix depending on the orthogonal transform shape. However, defining individual quantization matrices for all orthogonal transform shapes would unnecessarily increase the amount of code for the quantization matrices. This invention provides a technology for generating quantization matrices compatible with rectangular orthogonal transforms. [Means for solving the problem]

[0007] One aspect of the present invention is an image coding device for coding an image, the image coding device including: a generating means for generating a second quantization matrix having a size of 4×16 (the 4×16 size indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; and a quantization means for quantizing transform coefficients in a sub-block of a size corresponding to the size of 4×16 using the second quantization matrix, wherein the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), and the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3), and the generating means at least (a) generates the second quantization matrix from the first quantization matrix having the size of 8×8. and (b) directly reducing the elements of the first quantization matrix having a size of 8×8 in the horizontal direction by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix in the horizontal direction, and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the vertical direction by arranging an element in an mth row and 2nth column (m and n are integers equal to or greater than 0) of the first quantization matrix as an element in a 2mth row and nth column of the second quantization matrix and an element in a (2m+1)th row and nth column of the second quantization matrix, thereby generating the second quantization matrix, and the generating means is capable of generating a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix. [Effects of the Invention]

[0008] According to the configuration of the present invention, it is possible to provide a technique for generating a quantization matrix that corresponds to rectangular orthogonal transformation. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a block diagram showing an example of the functional configuration of an image encoding device. [Figure 2] FIG. 1 is a block diagram showing an example of the functional configuration of an image decoding device. [Figure 3] 10 is a flowchart of an encoding process of an input image and a quantization matrix. [Figure 4] 10 is a flowchart of a bitstream decoding process. [Figure 5] FIG. 1 is a block diagram showing an example of the hardware configuration of a computer device. [Figure 6] FIG. 10 is a diagram showing an example of the configuration of a bit stream. [Figure 7] FIG. 10 is a diagram showing an example of a division method. [Figure 8] FIG. 10 is a diagram showing an example of an original quantization matrix. [Figure 9] FIG. 10 is a diagram showing an example of a quantization matrix. [Figure 10] FIG. 10 is a diagram showing an example of a quantization matrix. [Figure 11] 10(a) is a diagram showing a scanning method, and FIG. 10(b) is a diagram showing a difference matrix 1000. FIG. [Figure 12] FIG. 10 is a diagram showing an example of the configuration of an encoding table. [Figure 13] FIG. 10 is a diagram showing an example of a quantization matrix. [Figure 14] FIG. 10 is a diagram showing an example of a quantization matrix. [Figure 15] FIG. 10 is a diagram showing an example of a quantization matrix. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the embodiment described below shows an example of a specific implementation of the present invention, and is one of the specific embodiments of the configuration described in the claims.

[0011] [First embodiment] First, an example of the functional configuration of an image encoding device according to this embodiment will be described using the block diagram in Fig. 1. A control unit 199 controls the operation of the entire image encoding device. A block division unit 102 divides an input image (an image of each frame constituting a moving image or a still image) into a plurality of basic blocks and outputs each of the divided basic blocks (divided images).

[0012] The generation unit 103 sets an N×N quantization matrix (a quantization matrix having N elements in the row direction and N elements in the column direction) as an original quantization matrix. Then, the generation unit 103 generates, from this original quantization matrix, a quantization matrix having a size (P×Q) corresponding to the number of pixels in a sub-block that serves as an orthogonal transform unit. Here, P and Q are positive integers, and P≠Q.

[0013] The prediction unit 104 divides each basic block into a plurality of sub-blocks (divided images). Then, for each sub-block, the prediction unit 104 performs intra-frame prediction or inter-frame prediction to generate a predicted image, and calculates the difference between the input image and the predicted image as a prediction error. The prediction unit 104 also generates prediction information, such as information indicating how the basic block was divided into sub-blocks, a prediction mode, and a motion vector, which are information necessary for prediction.

[0014] The transform / quantization unit 105 generates transform coefficients for each subblock by orthogonally transforming the prediction error for each subblock. Then, for each subblock, the transform / quantization unit 105 obtains a quantization matrix corresponding to the size of the subblock from the generation unit 103, and quantizes the transform coefficients in the subblock using the quantization matrix to generate quantized coefficients for the subblock.

[0015] The inverse quantization and inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated by the transform and quantization unit 105 using the quantization matrix used to quantize the sub-block, thereby generating transform coefficients, and then performs inverse orthogonal transform on the transform coefficients to generate prediction errors.

[0016] The image reproduction unit 107 generates a predicted image from the encoded image data stored in the frame memory 108 based on the prediction information generated by the prediction unit 104, and reproduces an image from the predicted image and the prediction error generated by the inverse quantization and inverse transform unit 106. The image reproduction unit 107 then stores the reproduced image in the frame memory 108. The image data stored in the frame memory 108 is used as image data to be referenced by the prediction unit 104 when making predictions for the image of the current frame or the next frame.

[0017] The in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the image stored in the frame memory 108 .

[0018] The encoding unit 110 generates encoded data by encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The quantization matrix encoding unit 113 generates encoded data by encoding the original quantization matrix.

[0019] The integrated coding unit 111 generates header code data using the coded data generated by the quantization matrix coding unit 113, and generates and outputs a bit stream including the coded data generated by the coding unit 110 and the header code data.

[0020] Next, the operation of the image encoding device according to this embodiment will be described. For ease of explanation, this embodiment will describe intra-prediction encoding of an input image, but the present invention is not limited to this and can also be applied to inter-prediction encoding of an input image. In this embodiment, the size of the quantization matrix of a basic block is set to a size having 16×16 elements, but the size of a basic block is not limited to 16×16. In this embodiment, the encoding process described above requires that a quantization matrix be prepared at the start of processing by the transform / quantization unit 105 and the inverse quantization / inverse transform unit 106.

[0021] First, generation and encoding of a quantization matrix will be described. An example of an original quantization matrix is shown in FIG. 8. The original quantization matrix 800 in FIG. 8 is an example of a quantization matrix corresponding to 8×8 orthogonal transformation coefficients. The method for determining the element value of each element constituting the original quantization matrix is not particularly limited. For example, a prescribed initial value may be used as the element value of each element constituting the original quantization matrix, or the element values of each element may be set individually. Also, the original quantization matrix may be generated according to the characteristics of the image.

[0022] In the present embodiment, the generation unit 103 generates a quantization matrix corresponding to the size of another orthogonal transformation (that is, the size of the sub-blocks to be divided) from such an original quantization matrix 800. In the present embodiment, the generation unit 103 generates rectangular quantization matrices 901, 902, 1001, 1002 shown in FIGS. 9(a), 9(b), 10(a), and 10(b) from the original quantization matrix 800 in FIG. 8. Note that for the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b), when the size of the quantization matrix is P×Q and the size of the original quantization matrix is N×N, P and Q are positive integers satisfying P < N < Q or Q < N < P.

[0023] The quantization matrix 901 shown in FIG. 9(a) is a quantization matrix corresponding to 8×16 orthogonal transformation coefficients. The quantization matrix 901 arranges the elements at the positions (row, column) = (2m, n) and (row, column) = (2m + 1, n) in the quantization matrix 901 (in the quantization matrix) with the elements at the position (row, column) = (m, n) in the original quantization matrix 800. Here, m and n are integers of 0 or more.

[0024] Quantization matrix 1001 shown in Fig. 10(a) is a quantization matrix corresponding to 16 × 8 orthogonal transform coefficients. Quantization matrix 1001 is obtained by arranging the element at position (row, column) = (m, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1001.

[0025] 9(b) is a quantization matrix corresponding to 4 × 16 orthogonal transform coefficients. In quantization matrix 902, the element at position (row, column) = (m, 2n) in original quantization matrix 800 is placed at position (row, column) = (2m, n) and position (row, column) = (2m+1, n) in quantization matrix 902.

[0026] Quantization matrix 1002 shown in Fig. 10(b) is a quantization matrix corresponding to 16 × 4 orthogonal transform coefficients. Quantization matrix 1002 is obtained by arranging the element at position (row, column) = (2m, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1002.

[0027] As described above, in this embodiment, the generation unit 103 generates and stores quantization matrices 901, 902, 1001, and 1002 from the original quantization matrix 800. As a result, regardless of whether the size of the divided rectangular sub-block is 8×16, 16×8, 16×4, or 4×16, a quantization matrix of the corresponding size has already been generated, allowing quantization / dequantization of the sub-block. In particular, the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b) are generated by enlarging the original quantization matrix in one direction and shrinking it in the other direction (thinning out every other row / column). This reduces the amount of data in the original quantization matrix compared to when it is generated by simply shrinking it from the original quantization matrix, and also suppresses image quality degradation compared to when it is generated by simply enlarging it from the original quantization matrix. Therefore, a balanced result between rate and distortion can be expected.

[0028] The generation unit 103 then stores the generated original quantization matrix 800 and quantization matrices 901, 902, 1001, and 1002 in two-dimensional form. It is also possible to store multiple quantization matrices for orthogonal transforms of the same size depending on the prediction method (described below), for example, whether intra-prediction or inter-prediction is used, or whether the encoding target is a luminance block or a chrominance block. Generally, a quantization matrix realizes quantization processing in accordance with human visual characteristics, so that, as shown in Figures 8 to 10, the elements in the low-frequency portion corresponding to the upper left part of the quantization matrix are small, and the elements in the high-frequency portion corresponding to the lower right part are large.

[0029] The quantization matrix encoding unit 113 acquires the quantization matrix stored in two-dimensional form from the generation unit 103, scans each element to calculate a difference, and generates a one-dimensional matrix (difference matrix) in which the differences are arranged one-dimensionally in the order of calculation.The quantization matrix encoding unit 113 then encodes the generated difference matrix to generate encoded data.

[0030] In this embodiment, the quantization matrices shown in FIGS. 9 and 10 can be generated from the original quantization matrix 800 as described above, and therefore the quantization matrix encoding unit 113 encodes only the original quantization matrix 800. Specifically, the difference between each element of the original quantization matrix 800 and the previous element is calculated in the scanning order, according to a scanning method that scans each element in the order indicated by the arrows in FIG. 11(a). For example, the original quantization matrix 800 is scanned by the diagonal scan shown in FIG. 11(a). After the first element "6" located in the upper left corner, the element "9" located immediately below it is scanned, and the difference "+3" is calculated. Furthermore, when encoding the first element of the original quantization matrix 800 ("6" in this embodiment), the difference from a predetermined initial value (e.g., "8") is calculated. However, this is not limiting, and the difference from any value, or the value of the first element itself, may also be used as the difference.

[0031] In this manner, in this embodiment, the quantization matrix encoding unit 113 generates the difference matrix 1000 shown in Figure 11(b) from the original quantization matrix 800 using the scanning method of Figure 11(a). The quantization matrix encoding unit 113 then encodes the difference matrix 1000 to generate encoded data. In this embodiment, encoding is performed using the encoding table shown in Figure 12(a), but the encoding table is not limited to this and, for example, the encoding table shown in Figure 12(b) may also be used.

[0032] In this embodiment, only the original quantization matrix 800 is coded out of the original quantization matrix 800 and the quantization matrices 901, 902, 1001, and 1002, so it is possible to reduce the amount of code for rectangular quantization matrices. Also, by coding using only one type of scanning method shown in Fig. 11(a) regardless of the shape of the quantization matrix to be generated, it is possible to limit the number of tables for implementing the scanning method of Fig. 11(a) to just one, thereby saving memory.

[0033] The integration coding unit 111 encodes header information necessary for decoding an image, which will be described later, and integrates the header information with the coded data of the quantization matrix generated by the quantization matrix coding unit 113 .

[0034] Next, the coding of an input image will be described. The block division unit 102 divides the input image into a plurality of basic blocks and outputs each of the divided basic blocks. The prediction unit 104 determines a division method for dividing the basic blocks into a plurality of sub-blocks. An example of a division method for dividing a basic block into a plurality of sub-blocks is shown in FIG. 7.

[0035] Fig. 7(b) shows a sub-block division method (quadtree division). That is, Fig. 7(b) shows a division method (division pattern) in which a basic block 700 is divided into two equal parts horizontally and vertically to divide the basic block 700 into four sub-blocks (each sub-block has a size of 8x8 pixels). On the other hand, Figs. 7(c) to (f) show sub-block division methods in which the basic block 700 is divided into rectangular sub-blocks.

[0036] FIG. 7(c) shows a division method in which a basic block 700 is divided into two equal parts vertically, thereby dividing the basic block 700 into two vertically elongated sub-blocks (each sub-block is 8x16 pixels in size).

[0037] FIG. 7(d) shows a division method in which a basic block 700 is divided into two equal parts horizontally, thereby dividing the basic block 700 into two horizontally elongated sub-blocks (each sub-block has a size of 16×8 pixels).

[0038] Figure 7(e) shows three vertically elongated sub-blocks obtained by dividing the basic block 700 vertically (from the left, a sub-block of 4x16 pixels, a sub-block of 8x16 pixels, and a sub-block of 4x16 pixels) (ternary tree division).

[0039] Figure 7(f) shows three horizontally elongated sub-blocks obtained by dividing the basic block 700 horizontally (a sub-block of 16x4 pixels, a sub-block of 16x8 pixels, and a sub-block of 16x4 pixels from the top) (ternary tree division).

[0040] In this embodiment, one of the division methods (division patterns) shown in Figures 7(b) to 7(f) is used, but is not limited to Figures 7(b) to 7(f). Note that, as shown in Figure 7(a), the basic block 700 may be divided into sub-blocks without being divided.

[0041] The prediction unit 104 also determines an intra-prediction mode, such as horizontal prediction or vertical prediction, for each sub-block. Then, for each basic block, the prediction unit 104 divides the basic block into multiple sub-blocks according to the determined division method. Then, for each sub-block, the prediction unit 104 generates a predicted image by performing prediction according to the determined intra-prediction mode using the image in the frame memory 108, and calculates the difference between the input image and the predicted image as a prediction error. The prediction unit 104 also generates prediction information, such as information indicating the sub-block division method (division pattern), the intra-prediction mode, and motion vectors, which are necessary for prediction.

[0042] The transform / quantization unit 105 generates transform coefficients for each sub-block by orthogonally transforming the prediction error for each sub-block. Then, for each sub-block, the transform / quantization unit 105 obtains a quantization matrix corresponding to the size of the sub-block from the generation unit 103, and quantizes the transform coefficients of the sub-block using the quantization matrix to generate quantized coefficients for the sub-block.

[0043] When quantizing the transform coefficients of an 8 pixel x 8 pixel sub-block, the transform / quantization unit 105 obtains the original quantization matrix 800 from the generation unit 103 and quantizes the transform coefficients using the obtained original quantization matrix 800.

[0044] When quantizing the transform coefficients of a sub-block of 8 pixels by 16 pixels, the transform / quantization unit 105 acquires a quantization matrix 901 from the generation unit 103 and quantizes the transform coefficients using the acquired quantization matrix 901.

[0045] When quantizing the transform coefficients of a 16 pixel x 8 pixel sub-block, the transform / quantization unit 105 obtains a quantization matrix 1001 from the generation unit 103 and quantizes the transform coefficients using the obtained quantization matrix 1001.

[0046] When quantizing the transform coefficients of a 4 pixel x 16 pixel sub-block, the transform / quantization unit 105 acquires a quantization matrix 902 from the generation unit 103 and quantizes the transform coefficients using the acquired quantization matrix 902.

[0047] When quantizing the transform coefficients of a 16 pixel x 4 pixel sub-block, the transform / quantization unit 105 obtains the quantization matrix 1002 from the generation unit 103 and quantizes the transform coefficients using the obtained quantization matrix 1002.

[0048] The inverse quantization and inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated by the transform and quantization unit 105 using the quantization matrix used to quantize the sub-block, thereby generating transform coefficients, and then performs inverse orthogonal transform on the transform coefficients to generate prediction errors.

[0049] The image reproduction unit 107 generates a predicted image from the processed image stored in the frame memory 108 based on the prediction information generated by the prediction unit 104, and reproduces an image from the predicted image and the prediction error generated by the inverse quantization and inverse transform unit 106. The image reproduction unit 107 then stores the reproduced image in the frame memory 108.

[0050] The in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the image stored in the frame memory 108 .

[0051] The encoding unit 110 generates encoded data by entropy encoding the quantized coefficients (quantized transform coefficients) generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The entropy encoding method is not limited to a specific method, and Golomb encoding, arithmetic encoding, Huffman encoding, etc. can be used.

[0052] The integrated coding unit 111 generates header coded data using the coded data generated by the quantization matrix coding unit 113, and generates a bitstream by multiplexing the header coded data with the coded data generated by the coding unit 110. The integrated coding unit 111 then outputs the generated bitstream.

[0053] An example of the bitstream structure is shown in Figure 6(a). The sequence header contains coded data corresponding to the original quantization matrix 800 and is composed of the coded results of each element. However, the coding position is not limited to this, and coding may be performed in a picture header or other header. Furthermore, when changing the quantization matrix within a single sequence, it is also possible to update it by newly coding the quantization matrix. In this case, all quantization matrices may be rewritten, or it is also possible to change only a part of them by specifying the transform block size of the quantization matrix to be rewritten.

[0054] The encoding process of an input image and a quantization matrix performed by the image encoding device described above will now be described with reference to the flowchart in Fig. 3. In step S301, generation unit 103 generates an original quantization matrix 800, and generates the above-mentioned quantization matrices 901, 902, 1001, and 1002 from the original quantization matrix 800.

[0055] 11(a) to calculate the difference, and generates a one-dimensional matrix in which the differences are arranged one-dimensionally in the order of calculation as a difference matrix 1000. The quantization matrix encoding unit 113 then encodes the generated difference matrix 1000 to generate encoded data.

[0056] In step S303, the integrated encoding unit 111 encodes header information necessary for encoding the image, and integrates the header information with the encoded data of the quantization matrix generated by the quantization matrix encoding unit 113.

[0057] In step S304, the block dividing unit 102 divides the input image into a plurality of basic blocks.

[0058] In step S305, the prediction unit 104 selects one of the basic blocks divided in step S304 that has not yet been selected as a selected basic block. The prediction unit 104 then determines a sub-block division method (division pattern) and divides the selected basic block into multiple sub-blocks according to the determined sub-block division method. The prediction unit 104 also determines an intra-prediction mode for each sub-block. For each sub-block, the prediction unit 104 generates a predicted image by using an image in the frame memory 108 to perform prediction according to the determined intra-prediction mode, and calculates the difference between the input image and the predicted image as a prediction error. The prediction unit 104 also generates prediction information, including information indicating the sub-block division method, the intra-prediction mode, and a motion vector, all of which are necessary for prediction.

[0059] In step S306, the transform / quantization unit 105 generates transform coefficients for each sub-block by orthogonally transforming the prediction error for each sub-block. Then, for each sub-block, the transform / quantization unit 105 quantizes the transform coefficients of the sub-block using a quantization matrix corresponding to the size of the sub-block from among the quantization matrices generated in step S301, to generate quantized coefficients for the sub-block.

[0060] In step S307, the inverse quantization and inverse transform unit 106 inverse quantizes the quantized coefficients of each sub-block generated in step S306 using the quantization matrix used in quantizing the sub-block to generate transform coefficients, and then performs inverse orthogonal transform on the generated transform coefficients to generate prediction errors.

[0061] In step S308, the image reproduction unit 107 generates a predicted image from the processed image stored in the frame memory 108, based on the prediction information generated by the prediction unit 104. The image reproduction unit 107 then reproduces an image from the predicted image and the prediction error generated by the inverse quantization and inverse transform unit 106. The image reproduction unit 107 then stores the reproduced image in the frame memory 108.

[0062] In step S309, the encoding unit 110 generates encoded data by entropy encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The integrated encoding unit 111 generates header encoded data using the encoded data generated by the quantization matrix encoding unit 113, and multiplexes the encoded data generated by the encoding unit 110 with the header encoded data to generate a bitstream.

[0063] In step S310, the control unit 199 determines whether all basic blocks have been coded. If the result of this determination is that all basic blocks have been coded, the process proceeds to step S311, and if there are any basic blocks that have not yet been coded, the process proceeds to step S305.

[0064] In step S311, the in-loop filter unit 109 performs in-loop filtering, such as deblocking filtering and sample adaptive offset, on the image stored in the frame memory .

[0065] According to the present embodiment described above, the amount of code for the quantization matrix can be reduced by generating multiple quantization matrices from one quantization matrix in step S301 and encoding only one quantization matrix in step S302. As a result, the overall data volume of the generated bitstream is reduced, thereby improving compression efficiency. Furthermore, the quantization matrices shown in FIGS. 9(b) and 10(b) are generated by enlarging the original quantization matrix in one direction and shrinking it in the other direction. This reduces the amount of data for the original quantization matrix compared to when it is generated by simply shrinking it, and also suppresses image quality degradation compared to when it is generated by simply enlarging it. Therefore, a balanced rate-distortion result can be expected.

[0066] In this embodiment, multiple quantization matrices are generated from one quantization matrix. However, it is also possible to select whether to generate each quantization matrix from a different quantization matrix or to encode each element, and to encode an identifier indicating the selection result in the header. For example, information indicating whether each element of the quantization matrix is ​​generated from another quantization matrix or encoded individually may be encoded in the header as quantization matrix encoding method information code, and the bitstream shown in FIG. 6(b) may be generated. This makes it possible to selectively generate a bitstream that prioritizes image quality control according to the size of the sub-block, or a bitstream with a smaller amount of quantization matrix code.

[0067] Furthermore, in this embodiment, a method has been described for generating quantization matrices 901, 902, 1001, and 1002 from original quantization matrix 800. However, the method for generating rectangular quantization matrices from square quantization matrices is not limited to the method described in the first embodiment.

[0068] For example, a method of generating rectangular quantization matrices 1201 to 1204 shown in FIGS. 13(a) and 13(b) and 14(a) and 14(b), respectively, from original quantization matrix 800 may be employed.

[0069] Quantization matrix 1201 shown in Figure 13(a) is a quantization matrix corresponding to 8 x 16 orthogonal transform coefficients. Each row of original quantization matrix 800 is arranged in an even-numbered row of quantization matrix 1201. In the odd-numbered rows of quantization matrix 1201, elements interpolated from the even-numbered rows on either side of the odd-numbered row are arranged. For example, element "14" at position (row, column) = (1, 2) in quantization matrix 1201 is obtained as a value interpolated from element "13" in the even-numbered row immediately above it and element "15" in the even-numbered row immediately below it.

[0070] Quantization matrix 1202 shown in Fig. 13(b) is a quantization matrix corresponding to 4 × 16 orthogonal transform coefficients. Quantization matrix 1202 is made up of only the even-numbered columns of quantization matrix 1201 generated from original quantization matrix 800 as described above, excluding the odd-numbered columns.

[0071] Quantization matrix 1203 shown in Figure 14(a) is a quantization matrix corresponding to 16x8 orthogonal transform coefficients. Each column of original quantization matrix 800 is arranged in an even-numbered column of quantization matrix 1203. In odd-numbered columns of quantization matrix 1203, elements interpolated from even-numbered columns on either side of the odd-numbered column are arranged. For example, element "14" at position (row, column) = (2,1) in quantization matrix 1203 is obtained as a value interpolated from element "13" in the adjacent column to the left and element "15" in the adjacent column to the right.

[0072] Quantization matrix 1204 shown in Fig. 14(b) is a quantization matrix corresponding to 16 × 4 orthogonal transform coefficients. Quantization matrix 1204 is made up of only the even rows of quantization matrix 1203 generated from original quantization matrix 800 as described above, excluding the odd rows.

[0073] In this way, another quantization matrix is ​​generated by expanding or contracting the original quantization matrix 800 in the row or column direction. This generation method allows for more precise control of quantization according to frequency components, resulting in improved image quality.

[0074] Furthermore, in this embodiment, elements in the even-numbered columns and rows of the original quantization matrix 800 are used when generating the quantization matrix 902 and the quantization matrix 1002. However, elements in the odd-numbered columns and rows of the original quantization matrix 800 may also be used to generate the quantization matrix.

[0075] For example, a quantization matrix 1601 corresponding to 4×16 orthogonal transform coefficients as shown in FIG. 15(a) or a quantization matrix 1602 corresponding to 16×4 orthogonal transform coefficients as shown in FIG. 15(b) may be generated.

[0076] Quantization matrix 1601 is obtained by placing the element at position (row, column) = (m, 2n+1) in original quantization matrix 800 at position (row, column) = (2m, n) and position (row, column) = (2m+1, n) in quantization matrix 1601.

[0077] Quantization matrix 1602 is obtained by placing the element at position (row, column) = (2m+1, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1602.

[0078] In this way, it is possible to generate multiple quantization matrices with the same shape but different elements from one original quantization matrix. Flags indicating the selection of even or odd columns of the original quantization matrix, and flags indicating the selection of even or odd rows, may be coded.

[0079] Furthermore, a configuration may be adopted in which multiple quantization matrices of the same shape generated in this manner are switched depending on the type of orthogonal transform method used. For example, if the orthogonal transform is based on discrete cosine transform, quantization matrix 902 or quantization matrix 1002 is generated, and if the orthogonal transform is based on discrete sine transform, quantization matrix 1601 or quantization matrix 1602 is generated. This makes it possible to use a quantization matrix that suits the properties of the orthogonal transform used.

[0080] Furthermore, if there are limitations on the size of the orthogonal transform to be used, the generation and encoding of some quantization matrices may be omitted. For example, if orthogonal transform is not performed on pixels of 4×16 or 16×4 sub-blocks, the generation of quantization matrix 902 and quantization matrix 1002 may be omitted. Furthermore, if orthogonal transform is not performed on pixels of sub-blocks of any of the sizes 8×8, 16×8, 8×16, 16×4, and 4×16, the encoding of original quantization matrix 800 itself may be omitted. In this case, the generation of quantization matrices 901, 902, 1001, and 1002 is naturally omitted. However, even if orthogonal transform is not used on pixels of 8×8 sub-blocks, if orthogonal transform is performed on pixels of sub-blocks of at least one of the sizes 16×8, 8×16, 16×4, and 4×16, original quantization matrix 800 is coded, and a quantization matrix corresponding to the orthogonal transform is generated. This allows only the quantization matrices that are used to be coded and generated, thereby reducing the amount of unnecessary coding when the quantization matrices are not used.

[0081] [Second embodiment] In this embodiment, an image decoding device that decodes an input image encoded by the image encoding device according to the first embodiment will be described. Differences from the first embodiment will be described below, and unless otherwise specified below, the present embodiment is the same as the first embodiment. An example of the functional configuration of the image decoding device according to this embodiment will be described using the block diagram of FIG. 2.

[0082] The control unit 299 controls the operation of the entire image decoding device. The separate decoding unit 202 acquires a bitstream generated by the image encoding device, separates coded data related to coefficients and information related to the decoding process from the bitstream, and decodes the coded data present in the header of the bitstream. In this embodiment, the separate decoding unit 202 separates coded data of the quantization matrix from the bitstream. In other words, the separate decoding unit 202 performs the reverse operation of the integrated encoding unit 111 described above.

[0083] The quantization matrix decoding unit 209 obtains the coded data of the quantization matrix separated from the bitstream by the separation decoding unit 202, and generates a quantization matrix by decoding the obtained coded data.

[0084] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the coded data separated from the bitstream by the separate decoding unit 202. The inverse quantization and inverse transform unit 204 performs the same operation as the inverse quantization and inverse transform unit 106 of the above-mentioned image coding device. The inverse quantization and inverse transform unit 204 obtains transform coefficients by performing inverse quantization on the quantization coefficients based on the quantization matrix decoded by the quantization matrix decoding unit 209, and obtains prediction errors by performing inverse orthogonal transform on the transform coefficients.

[0085] The image reproduction unit 205 generates a predicted image by referring to the image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. The image reproduction unit 205 then generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization and inverse transform unit 204, and stores the generated reproduced image in the frame memory 206.

[0086] The in-loop filter unit 207 performs in-loop filtering such as deblocking filtering on the reconstructed image stored in the frame memory 206. The reconstructed image stored in the frame memory 206 is output as appropriate by the control unit 299. The output destination of the reconstructed image is not limited to a specific output destination, and for example, the reconstructed image may be displayed on the display screen of a display device such as a monitor, or the reconstructed image may be output to a projection device such as a projector.

[0087] Next, the operation of the image decoding device having the above configuration (bitstream decoding process) will be described. In this embodiment, the bitstream input to the separation decoding unit 202 is a bitstream for each frame of a moving image, but it may also be a bitstream of a still image. Furthermore, for ease of explanation, this embodiment will only describe intra-prediction decoding process, but the present invention is not limited to this and can also be applied to inter-prediction decoding process.

[0088] The separate decoding unit 202 obtains a bitstream for one frame generated by the image encoding device, separates coded data related to information about the decoding process and coefficients from the bitstream, and decodes the coded data present in the bitstream header. The separate decoding unit 202 extracts quantization matrix coded data (data about the quantization matrix) from the sequence header of the bitstream in Figure 6(a) and sends the quantization matrix coded data to the quantization matrix decoding unit 209. The separate decoding unit 202 also outputs coded data of picture data in units of basic blocks to the decoding unit 203.

[0089] The quantization matrix decoding unit 209 generates a one-dimensional difference matrix by decoding the coded data of the quantization matrix separated from the bitstream by the separation decoding unit 202. To decode the coded data of the quantization matrix, the coding table used on the coding side (in the first embodiment, the coding table used on the coding side, either the coding table shown in FIG. 12(a) or the coding table shown in FIG. 12(b)) is used. The quantization matrix decoding unit 209 then generates a two-dimensional quantization matrix (original quantization matrix) by inversely scanning the generated one-dimensional difference matrix. This process is the reverse of the process performed by the quantization matrix coding unit 113 described in the first embodiment. That is, in this embodiment, the difference matrix 1000 shown in FIG. 11(b) is used to generate the original quantization matrix 800 shown in FIG. 8 using the scanning method shown in FIG. 11(a). The quantization matrix decoding unit 209 then generates quantization matrices 901, 902, 1001, and 1002 from the generated original quantization matrix 800, in the same manner as in the first embodiment. At this time, if necessary, the flags indicating the selection of even or odd columns and the flags indicating the selection of even or odd rows of the original quantization matrix are decoded. Then, generation from the original quantization matrix is ​​performed according to these flags.

[0090] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the coded data separated from the bitstream by the separate decoding unit 202. The inverse quantization and inverse transform unit 204 selects a quantization matrix corresponding to the size of the sub-block to be decoded from the original quantization matrix 800 and quantization matrices 901, 902, 1001, and 1002 generated by the quantization matrix decoding unit 209, based on information indicating the sub-block division pattern, which is one piece of prediction information. The inverse quantization and inverse transform unit 204 then uses the selected quantization matrix to perform inverse quantization on the quantized coefficients of the sub-block to be decoded, thereby obtaining transform coefficients, and performs inverse orthogonal transform on the transform coefficients, thereby obtaining prediction errors.

[0091] The image reproduction unit 205 generates a predicted image by referring to an image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. The image reproduction unit 205 then generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization and inverse transform unit 204, and stores the generated reproduced image in the frame memory 206. The in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reproduced image stored in the frame memory 206.

[0092] The decoding process of one frame of a bitstream performed by the image decoding device described above will be described with reference to the flowchart in Fig. 4. In step S401, the separate decoding unit 202 obtains one frame of a bitstream generated by the image coding device. The separate decoding unit 202 then separates coded data related to information about the decoding process and coefficients from the bitstream, and decodes the coded data present in the header of the bitstream.

[0093] In step S402, the quantization matrix decoding unit 209 generates a one-dimensional difference matrix by decoding the coded data of the quantization matrix separated from the bitstream by the separation decoding unit 202. The quantization matrix decoding unit 209 then inversely scans the generated one-dimensional difference matrix to generate a two-dimensional original quantization matrix 800. The quantization matrix decoding unit 209 then generates quantization matrices 901, 902, 1001, and 1002 from the generated original quantization matrix 800 in the same manner as in the first embodiment. In particular, the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b) are generated by enlarging the original quantization matrix in one direction and shrinking it in the other direction. This reduces the amount of data for the original quantization matrix compared to when it is generated by simply shrinking it from the original quantization matrix, and also suppresses image quality degradation compared to when it is generated by simply enlarging it from the original quantization matrix. Therefore, a balanced result in terms of rate and distortion can be expected.

[0094] Note that when the sizes of the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b) are set to P×Q, which is the size of the quantization matrix generated when the size of the original quantization matrix is N×N, P and Q are positive integers satisfying P < N < Q or Q < N < P. Note that when dividing a block into sub-blocks, it may be adaptively generated according to the division information indicating which type of pattern among FIGS. 7(b) to 7(f) is used.

[0095] The processes of steps S403 to S406 are performed for each basic block in the input image. In step S403, the decoding unit 203 obtains quantization coefficients and prediction information by decoding the encoded data separated from the bit stream by the separation decoding unit 202.

[0096] In step S404, the inverse quantization / inverse transformation unit 204 selects a quantization matrix corresponding to the size and shape of the sub-block to be decoded from the quantization matrices generated by the quantization matrix decoding unit 209. Then, the inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the quantization coefficients of the sub-block to be decoded using the selected quantization matrix, and obtains a prediction error by performing an inverse orthogonal transformation on the transformation coefficients.

[0097] In step S405, the image reproduction unit 205 generates a predicted image by referring to the image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. Then, the image reproduction unit 205 generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transformation unit 204, and stores the generated reproduced image in the frame memory 206.

[0098] In step S406, the control unit 299 determines whether decoding of all basic blocks included in the bitstream has been completed. If this determination shows that decoding of all basic blocks included in the bitstream has been completed, the process proceeds to step S407. On the other hand, if there are sub-blocks among all basic blocks included in the bitstream that have not yet been decoded, the process from step S403 onwards is repeated for those sub-blocks that have not yet been decoded.

[0099] In step S407, the in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206.

[0100] According to the present embodiment described above, only the original quantization matrix generated in the first embodiment is coded, so that it is possible to decode a bitstream with a reduced amount of coding for the quantization matrix. Also, since the element scanning method used to decode the quantization matrix is ​​limited to one type shown in Fig. 11(a), the number of tables for implementing the scanning method shown in Fig. 11(a) is limited to only one, thereby saving memory.

[0101] In this embodiment, multiple quantization matrices are generated from one quantization matrix. However, an identifier may be decoded from the header to select whether each quantization matrix is ​​generated by deriving it from another quantization matrix or whether each element coded independently of another quantization matrix is ​​decoded. For example, information indicating whether each element of a quantization matrix is ​​generated from another quantization matrix or decoded individually may be decoded as quantization matrix coding method information code from the header of the bitstream in FIG. 6(b). This allows a bitstream to be decoded that selects whether to prioritize image quality control according to the size of the subblock or to reduce the amount of code for the quantization matrix.

[0102] Furthermore, in this embodiment, a method for generating quantization matrices 901, 902, 1001, and 1002 from original quantization matrix 800 has been described. However, as with the image encoding device, the method for generating a rectangular quantization matrix from a square quantization matrix is ​​not limited to the method described in the first embodiment. For example, a method for generating quantization matrices 1201 to 1204 shown in FIGS. 13(a) and 13(b) and 14(a) and 14(b), respectively, from original quantization matrix 800 may also be employed. This allows for more precise quantization control according to frequency components, resulting in a decoded bitstream with improved image quality.

[0103] Furthermore, if a certain quantization matrix size is not used due to limitations on the size of the orthogonal transform used, decoding and generation of the unused quantization matrix may be omitted. For example, if orthogonal transform is not performed on sub-blocks of 4×16 or 16×4 pixel sizes, generation of quantization matrix 902 and quantization matrix 1002 may be omitted. Furthermore, if orthogonal transform is not performed on sub-blocks of 8×8, 16×8, 8×16, 16×4, or 4×16 pixel sizes, decoding of original quantization matrix 800 itself may be omitted. In this case, generation of quantization matrices 901, 902, 1001, and 1002 is naturally omitted. However, even if orthogonal transform is not performed on sub-blocks of 8×8 pixel size, if orthogonal transform is performed on sub-blocks of at least one rectangular size of 16×8, 8×16, 16×4, or 4×16, original quantization matrix 800 is decoded and a quantization matrix corresponding to the orthogonal transform is generated. This allows decoding and generating only the quantization matrices that are used, and enables decoding of a bitstream that reduces the amount of unnecessary coding that would be required if a quantization matrix were not used.

[0104] In this embodiment, the image decoding device has been described as being separate from the image encoding device according to the first embodiment, but the image decoding device and the image encoding device according to the first embodiment may be integrated into a single device. In this case, this device can encode an input image and decode the encoded input image as necessary.

[0105] [Third embodiment] 1 and 2 may be implemented entirely as hardware, or some of them may be implemented as software (computer programs). In that case, a computer device that has frame memory 108 or frame memory 206 as a memory device and is capable of executing the computer programs can be applied to the above-mentioned image encoding device or image decoding device. An example of the hardware configuration of a computer device that can be applied to the above-mentioned image encoding device or image decoding device will be described using the block diagram of FIG.

[0106] The CPU 501 executes various processes using computer programs and data stored in the RAM 502 and the ROM 503. As a result, the CPU 501 controls the operation of the entire computer device, and also executes or controls the processes described above as being performed by the image encoding device and the image decoding device.

[0107] The RAM 502 has an area for storing computer programs and data loaded from the ROM 503 or external storage device 506, and data received from the outside via an I / F (interface) 507 (for example, the above-mentioned moving image and still image data). The RAM 502 also has a work area used by the CPU 501 when executing various processes. In this way, the RAM 502 can provide various areas as needed. The ROM 503 stores setting data, startup programs, etc. for the computer device.

[0108] The operation unit 504 is a user interface such as a keyboard, a mouse, or a touch panel, and can be operated by the user to input various instructions to the CPU 501.

[0109] The display unit 505 is configured with a liquid crystal screen, a touch panel screen, or the like, and can display the processing results of the CPU 501 as images, text, etc. For example, a reproduced image decoded by the image decoding device described above may be displayed on the display unit 505. The display unit 505 may also be a projection device such as a projector that projects images and text.

[0110] The external storage device 506 is a large-capacity information storage device such as a hard disk drive. The external storage device 506 stores an OS (operating system), computer programs and data for causing the CPU 501 to execute or control the various processes described above as being performed by the image encoding device and the image decoding device. The computer programs stored in the external storage device 506 include computer programs for causing the CPU 501 to realize the functions of each functional unit other than the frame memory 108 and the frame memory 206. The data stored in the external storage device 506 also includes various information required for encoding and decoding, such as the encoding tables shown in Figures 12(a) and 12(b).

[0111] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501, and are then processed by the CPU 501. The frame memory 108 and frame memory 206 can be implemented by memory devices such as the RAM 502, the ROM 503, and the external storage device 506.

[0112] The I / F 507 functions as an interface for data communication with external devices. For example, moving images and still images can be acquired from an external server device or imaging device via the I / F 507 into the RAM 502 or the external storage device 506.

[0113] The CPU 501, RAM 502, ROM 503, operation unit 504, display unit 505, external storage device 506, and I / F 507 are all connected to a bus 508. Note that the configuration shown in Fig. 5 is merely an example of the hardware configuration of a computer device applicable to the above-mentioned image encoding device and image decoding device, and various changes / modifications are possible.

[0114] [Fourth embodiment] The numerical values ​​used in the above description are used for the purpose of concrete explanation, and it is not intended that the above embodiments be limited to the numerical values ​​used. For example, the size of the quantization matrix and each element of the quantization matrix are not limited to the numerical values ​​described above.

[0115] In addition, some or all of the above-described embodiments may be used in appropriate combination, and some or all of the above-described embodiments may be selectively used.

[0116] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0117] 102: Block division unit 103: Generation unit 104: Prediction unit 105: Transformation and quantization unit 106: Inverse quantization and inverse transformation unit 107: Image reproduction unit 108: Frame memory 109: In-loop filter unit 110: Encoding unit 111: Integrated encoding unit 113: Quantization matrix encoding unit

Claims

1. An image encoding device that encodes an image, comprising: a generating means for generating a second quantization matrix having a size of 4x16 (the size of 4x16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8x8; a quantization means for quantizing transform coefficients in a sub-block of a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); The generating means includes at least (a) when generating the second quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the horizontal direction by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the horizontal direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the vertical direction by arranging an element at m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element at 2m rows and n columns of the second quantization matrix and an element at (2m+1) rows and n columns of the second quantization matrix; generating the second quantization matrix; The generating means is capable of generating a third quantization matrix of a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix. An image encoding device comprising:

2. Furthermore, encoding means for encoding the first quantization matrix; means for outputting a bitstream including encoded data of the image and encoded data of the first quantization matrix; The image encoding device according to claim 1 , further comprising:

3. When a transform process is performed on a sub-block of a size corresponding to the 4×16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​executable, 2. The image encoding device according to claim 1, wherein when a transform process is not performed on a sub-block of a size corresponding to the 4x16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​not performed.

4. 2. The image encoding device according to claim 1, wherein the sub-block of a size corresponding to the 4x16 size corresponds to the first or third sub-block of three sub-blocks obtained by vertically dividing a 16x16 block into thirds, the first sub-block of the 4x16 size, the second sub-block of the 8x16 size, and the third sub-block of the 4x16 size.

5. Furthermore, The generating means generating a fourth quantization matrix having a size of 16x4 (the size of 16x4 indicates a size of 16 in the horizontal direction and a size of 4 in the vertical direction) from the first quantization matrix having a size of 8x8; the fourth quantization matrix has an s-th row and a t-th column (where s is an integer satisfying 0≦s≦3 and t is an integer satisfying 0≦t≦15); The generating means includes at least (a) when generating the fourth quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the vertical direction by excluding all elements in the first, third, fifth, and seventh rows of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the vertical direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the horizontal direction by arranging an element of 2m rows and n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element of m rows and 2n columns in the fourth quantization matrix and an element of m rows and (2n+1) columns in the fourth quantization matrix, generating the fourth quantization matrix; The quantization means quantizes the transform coefficients in the sub-blocks having a size corresponding to the 16×4 size using the fourth quantization matrix.

2. The image encoding device according to claim 1.

6. 6. The image encoding device according to claim 5, characterized in that when conversion processing is not performed on sub-blocks of a size corresponding to the 4x16 size and sub-blocks of a size corresponding to the 16x4 size, the processing of generating the second quantization matrix from the first quantization matrix and the processing of generating the fourth quantization matrix from the first quantization matrix are not performed.

7. 6. The image coding device according to claim 5, wherein the sub-block of a size corresponding to the 16x4 size corresponds to the first or third sub-block of three sub-blocks obtained by horizontally dividing a 16x16 block into thirds, the first sub-block of the 16x4 size, the second sub-block of the 16x8 size, and the third sub-block of the 16x4 size.

8. The encoding means includes at least (a) encoding a difference value between the element in the 0th row and 0th column of the first quantization matrix and a predetermined value; (b) encoding a difference value between the element at row 1, column 0 of the first quantization matrix, which is the element next to the element at row 0, column 0 in a predetermined scanning order, and the element at row 0, column 0 of the first quantization matrix; (c) encoding a difference value between the element in the 0th row and 1st column of the first quantization matrix, which is the element next to the element in the 1st row and 0th column in the predetermined scanning order, and the element in the 1st row and 0th column of the first quantization matrix; (d) encoding a difference value between the element in the second row and the zeroth column of the first quantization matrix, which is the element next to the element in the zeroth row and the first column in the predetermined scanning order, and the element in the zeroth row and the first column of the first quantization matrix; Encoding the first quantization matrix 3. The image encoding device according to claim 2.

9. An image decoding device that decodes an encoded image, comprising: a generating means for generating a second quantization matrix having a size of 4x16 (the size of 4x16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8x8; and an inverse quantization means for inversely quantizing the quantized transform coefficients in the sub-blocks having a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); The generating means includes at least (a) when generating the second quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the horizontal direction by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the horizontal direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the vertical direction by arranging an element at m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element at 2m rows and n columns of the second quantization matrix and an element at (2m+1) rows and n columns of the second quantization matrix; generating the second quantization matrix; The generating means is capable of generating a third quantization matrix of a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix.

1. An image decoding device comprising:

10. When a transform process is performed on a sub-block of a size corresponding to the 4×16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​executable, 10. The image decoding device according to claim 9, wherein when a transform process is not performed on a sub-block of a size corresponding to the 4x16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​not performed.

11. 10. The image decoding device of claim 9, wherein the sub-block of a size corresponding to the 4x16 size corresponds to the first sub-block or the third sub-block of three sub-blocks obtained by vertically dividing a 16x16 block into thirds, the first sub-block of the 4x16 size, the second sub-block of the 8x16 size, and the third sub-block of the 4x16 size.

12. Furthermore, The generating means generating a fourth quantization matrix having a size of 16x4 (the size of 16x4 indicates a size of 16 in the horizontal direction and a size of 4 in the vertical direction) from the first quantization matrix having a size of 8x8; the fourth quantization matrix has an s-th row and a t-th column (where s is an integer satisfying 0≦s≦3 and t is an integer satisfying 0≦t≦15); The generating means includes at least (a) when generating the fourth quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the vertical direction by excluding all elements in the first, third, fifth, and seventh rows of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the vertical direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the horizontal direction by arranging an element of 2m rows and n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element of m rows and 2n columns in the fourth quantization matrix and an element of m rows and (2n+1) columns in the fourth quantization matrix, generating the fourth quantization matrix; The inverse quantization means inversely quantizes the quantized transform coefficients in the sub-blocks having a size corresponding to the 16×4 size using the third quantization matrix.

10. The image decoding device according to claim 9.

13. 13. The image decoding device of claim 12, wherein when conversion processing is not performed on sub-blocks of a size corresponding to the 4x16 size and sub-blocks of a size corresponding to the 16x4 size, the processing of generating the second quantization matrix from the first quantization matrix and the processing of generating the third quantization matrix from the first quantization matrix are not performed.

14. 13. The image decoding device of claim 12, wherein the sub-block of a size corresponding to the 16x4 size corresponds to the first sub-block or the third sub-block of three sub-blocks obtained by horizontally dividing a 16x16 block into thirds, the first sub-block of a size of 16x4, the second sub-block of a size of 16x8, and the third sub-block of a size of 16x4.

15. Furthermore, at least (a) decoding coded data corresponding to the first quantization matrix to derive a one-dimensional matrix obtained from a plurality of difference values ​​including a difference value between the element in the 0th row and 0th column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in a predetermined scanning order; (b) deriving the element at row 0, column 0 of the first quantization matrix using a first element that is the first element in the one-dimensional matrix; (c) deriving the element in the 1st row and 0th column of the first quantization matrix, which is the element in the 0th row and 0th column next to the element in the 0th row and 0th column in a predetermined scanning order, using a second element that is the element next to the first element in the one-dimensional matrix; (d) deriving the element in the 0th row and the 1st column of the first quantization matrix, which is the element next to the element in the 1st row and the 0th column in the predetermined scanning order, using a third element that is the element next to the second element in the one-dimensional matrix; (e) deriving the element in the second row and the zeroth column of the first quantization matrix, which is the element next to the element in the zeroth row and the first column in the predetermined scanning order, using a fourth element that is the element next to the third element in the one-dimensional matrix; a decoding means for decoding the first quantization matrix; 10. The image decoding device according to claim 9.

16. 1. An image encoding method for encoding an image, comprising: a generating step of generating a second quantization matrix having a size of 4x16 (the size of 4x16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8x8; a quantization step of quantizing transform coefficients in sub-blocks of a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); In the producing step, at least (a) when generating the second quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the horizontal direction by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the horizontal direction; and (b) generating the second quantization matrix by vertically increasing the elements of the first quantization matrix having a size of 8×8 such that an element at m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix is ​​arranged as an element at 2m rows and n columns of the second quantization matrix and as an element at (2m+1) rows and n columns of the second quantization matrix; In the generating step, a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) can be generated from the first quantization matrix.

1. An image coding method comprising:

17. An image decoding method for decoding an encoded image, comprising: a generating step of generating a second quantization matrix having a size of 4x16 (the size of 4x16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8x8; a dequantization step of dequantizing the quantized transform coefficients in the sub-blocks of a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); In the producing step, at least (a) when generating the second quantization matrix from the first quantization matrix, directly reduce the elements of the first quantization matrix having a size of 8x8 in the horizontal direction by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8x8 in the horizontal direction; and (b) generating the second quantization matrix by vertically increasing the elements of the first quantization matrix having a size of 8×8 such that an element at m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix is ​​arranged as an element at 2m rows and n columns of the second quantization matrix and as an element at (2m+1) rows and n columns of the second quantization matrix; In the generating step, a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) can be generated from the first quantization matrix.

1. An image decoding method comprising:

18. A computer program for causing a computer to function as each of the means of the image encoding device according to any one of claims 1 to 8.

19. A computer program for causing a computer to function as each of the means of the image decoding device according to any one of claims 9 to 15.

Images