Image encoding device, image encoding method, image decoding device, and image decoding method

Abstract

To provide a technique to generate a quantization matrix corresponding to rectangular orthogonal transformation.SOLUTION: All elements on the first row, the third row, the fifth row, and the seventh row of a first quantization matrix are removed without increasing elements of the first quantization matrix of 8×8 size in a horizontal direction, thereby reducing the elements of the first quantization matrix of 8×8 size in the horizontal direction. Elements on the m-th column and the 2n-th row (m, n are an integer of 0 or more) in the first quantization matrix are arranged as elements on the 2m-th column and the n-th row and elements on the (2m+1)-th column and the n-th row of a second quantization matrix to increase in a vertical direction, the elements in the first quantization matrix of 8×8 size, thereby generating the second quantization matrix. A third quantization matrix of 8×16 size can be generated from the first quantization matrix.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to image encoding and decoding techniques. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter referred to as HEVC) is known as a coding method for compressing and recording moving images. In order to improve coding efficiency, HEVC employs basic blocks larger than conventional macroblocks (16 pixels x 16 pixels). These large basic blocks are called coding tree units (CTUs) and are up to 64 pixels x 64 pixels in size. CTUs are further divided into subblocks, which are units for prediction and transformation.

[0003] Furthermore, HEVC uses a matrix called a quantization matrix, which is used to weight coefficients after orthogonal transformation (hereinafter referred to as orthogonal transformation coefficients) according to frequency components. By further reducing data of high frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technology for encoding such a quantization matrix.

[0004] In recent years, activities have been started to internationally standardize a more efficient coding method as a successor to HEVC. JVET (Joint Video Experts Team) was established between ISO / IEC and ITU-T, and standardization is underway as the Versatile Video Coding (VVC) coding method (hereinafter referred to as VVC). In order to improve coding efficiency, rectangular sub-block based intra prediction and orthogonal transform methods are being considered in addition to the conventional square sub-block based intra prediction and orthogonal transform methods. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-38758 A Summary of the Invention [Problem to be solved by the invention]

[0006] In VVC, as in HEVC, the introduction of a quantization matrix is ​​being considered. Furthermore, in VVC, not only square but also rectangular subblock division and the corresponding orthogonal transform shape are being considered. Since the distribution of each orthogonal transform coefficient differs depending on the orthogonal transform shape, it is desirable to apply an optimal quantization matrix according to the orthogonal transform shape. However, if individual quantization matrices are defined for all orthogonal transform shapes, the amount of code of the quantization matrix increases unnecessarily. In the present invention, a technology for generating a quantization matrix corresponding to rectangular orthogonal transform is provided. [Means for solving the problem]

[0007] One aspect of the present invention is an image coding device that codes an image, the image coding device including: a generating means for generating a second quantization matrix having a size of 4×16 (the size of 4×16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; and a quantization means for quantizing transformation coefficients in a sub-block having a size corresponding to the size of 4×16 using the second quantization matrix, the first quantization matrix having an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix having a p-th row and a q-th column (p is an integer satisfying 0≦p≦15 and q is an integer satisfying 0≦q≦3), and the generating means at least: (a) generates the first quantization matrix having the size of 8×8 when generating the second quantization matrix from the first quantization matrix; and (b) directly reducing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction by excluding all elements in the 1st, 3rd, 5th, and 7th columns of the first quantization matrix without increasing the elements of the first quantization matrix in the horizontal direction, and (b) increasing the elements of the first quantization matrix having the size of 8×8 in the vertical direction by arranging an element in m rows and 2n columns (m and n are integers equal to or greater than 0) of the first quantization matrix as an element in 2m rows and n columns of the second quantization matrix and an element in (2m+1) rows and n columns of the second quantization matrix, thereby generating the second quantization matrix, wherein the generating means is capable of generating a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix. Effect of the Invention

[0008] According to the configuration of the present invention, it is possible to provide a technique for generating a quantization matrix compatible with rectangular orthogonal transformation. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a block diagram showing an example of the functional configuration of an image encoding device. [Diagram 2] FIG. 2 is a block diagram showing an example of the functional configuration of an image decoding device. [Diagram 3] 1 is a flowchart of an encoding process of an input image and a quantization matrix. [Figure 4] 13 is a flowchart of a bitstream decoding process. [Diagram 5] FIG. 2 is a block diagram showing an example of the hardware configuration of a computer apparatus. [Figure 6] FIG. 1 is a diagram showing an example of the configuration of a bit stream. [Figure 7] FIG. [Figure 8] FIG. 13 is a diagram showing an example of an original quantization matrix. [Figure 9] FIG. 13 is a diagram showing an example of a quantization matrix. [Figure 10] FIG. 13 is a diagram showing an example of a quantization matrix. [Figure 11] 10(a) is a diagram showing a scanning method, and (b) is a diagram showing a difference matrix 1000. [Figure 12] FIG. 4 is a diagram showing an example of the configuration of an encoding table. [Figure 13] FIG. 13 is a diagram showing an example of a quantization matrix. [Figure 14] FIG. 13 is a diagram showing an example of a quantization matrix. [Figure 15] FIG. 13 is a diagram showing an example of a quantization matrix. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the embodiment described below shows an example of a specific implementation of the present invention, and is one of the specific embodiments of the configuration described in the claims.

[0011] [First embodiment] First, an example of the functional configuration of the image encoding device according to this embodiment will be described with reference to the block diagram in Fig. 1. A control unit 199 controls the operation of the entire image encoding device. A block division unit 102 divides an input image (an image of each frame constituting a moving image or a still image) into a plurality of basic blocks, and outputs each of the divided basic blocks (divided images).

[0012] The generation unit 103 sets a quantization matrix having a size of N×N (a quantization matrix having N elements in the row direction and N elements in the column direction) as an original quantization matrix. Then, the generation unit 103 generates a quantization matrix having a size (P×Q) of pixels of a sub-block that is an orthogonal transform unit from this original quantization matrix. Here, P and Q are positive integers and P≠Q.

[0013] The prediction unit 104 divides each basic block into a plurality of sub-blocks (divided images). The prediction unit 104 then performs intra-frame prediction or inter-frame prediction for each sub-block to generate a predicted image, and obtains the difference between the input image and the predicted image as a prediction error. The prediction unit 104 also generates, as prediction information, information indicating how the basic block is divided into sub-blocks, a prediction mode, a motion vector, and other information required for prediction.

[0014] The transform / quantization unit 105 generates transform coefficients for each subblock by performing orthogonal transform on the prediction error for each subblock. Then, for each subblock, the transform / quantization unit 105 obtains a quantization matrix corresponding to the size of the subblock from the generation unit 103, and quantizes the transform coefficients in the subblock using the quantization matrix to generate quantized coefficients for the subblock.

[0015] The inverse quantization / inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated by the transform / quantization unit 105 using the quantization matrix used for quantizing the sub-block to generate transform coefficients, and then performs inverse orthogonal transform on the transform coefficients to generate prediction errors.

[0016] The image reproduction unit 107 generates a predicted image from the encoded image data stored in the frame memory 108 based on the prediction information generated by the prediction unit 104, and reproduces an image from the predicted image and the prediction error generated by the inverse quantization and inverse transform unit 106. The image reproduction unit 107 then stores the reproduced image in the frame memory 108. The image data stored in the frame memory 108 is image data that is referred to when the prediction unit 104 predicts an image of the current frame or the next frame.

[0017] The in-loop filter unit 109 performs in-loop filter processing such as deblocking filtering and sample adaptive offset on the image stored in the frame memory 108 .

[0018] The encoding unit 110 generates encoded data by encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The quantization matrix encoding unit 113 generates encoded data by encoding the original quantization matrix.

[0019] The integrated coding unit 111 generates header code data using the coded data generated by the quantization matrix coding unit 113, and generates and outputs a bit stream including the coded data generated by the coding unit 110 and the header code data.

[0020] Next, the operation of the image encoding device according to this embodiment will be described. In this embodiment, for ease of explanation, intra-prediction encoding processing of an input image will be described, but the present invention is not limited to this, and the present invention is also applicable to inter-prediction encoding processing of an input image. In this embodiment, the size of the quantization matrix of a basic block is set to a size having 16×16 elements, but the size of the basic block is not limited to 16×16. In this embodiment, in the above encoding processing, a quantization matrix needs to be prepared at the start of processing by the transform / quantization unit 105 and the inverse quantization / inverse transform unit 106.

[0021] First, the generation and encoding of the quantization matrix will be described. An example of the original quantization matrix is shown in FIG. 8. The original quantization matrix 800 in FIG. 8 is an example of a quantization matrix corresponding to 8×8 orthogonal transformation coefficients. The method for determining the element value of each element constituting the original quantization matrix is not particularly limited. For example, a prescribed initial value may be used as the element value of each element constituting the original quantization matrix, or the element values of the respective elements may be set individually. Further, the original quantization matrix may be generated according to the characteristics of the image.

[0022] In the present embodiment, the generation unit 103 generates a quantization matrix corresponding to the size of another orthogonal transformation (that is, the size of the sub-blocks to be divided) from such an original quantization matrix 800. In the present embodiment, the generation unit 103 generates rectangular quantization matrices 901, 902, 1001, and 1002 shown in FIGS. 9(a), 9(b), 10(a), and 10(b) from the original quantization matrix 800 in FIG. 8. Note that for the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b), when the size of the quantization matrix is P×Q, where P and Q are positive integers satisfying P < N < Q or Q < N < P when the size of the original quantization matrix is N×N.

[0023] The quantization matrix 901 shown in FIG. 9(a) is a quantization matrix corresponding to 8×16 orthogonal transformation coefficients. The quantization matrix 901 is obtained by arranging the element at the position (row, column) = (m, n) in the original quantization matrix 800 at the positions (row, column) = (2m, n) and (row, column) = (2m + 1, n) in the quantization matrix 901 (in the quantization matrix). Here, m and n are integers of 0 or more.

[0024] Quantization matrix 1001 shown in Fig. 10(a) is a quantization matrix corresponding to 16 × 8 orthogonal transform coefficients. Quantization matrix 1001 is obtained by arranging an element at position (row, column) = (m, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1001.

[0025] Quantization matrix 902 shown in Fig. 9(b) is a quantization matrix corresponding to 4 × 16 orthogonal transform coefficients. Quantization matrix 902 is obtained by arranging an element at position (row, column) = (m, 2n) in original quantization matrix 800 at position (row, column) = (2m, n) and position (row, column) = (2m+1, n) in quantization matrix 902.

[0026] Quantization matrix 1002 shown in Fig. 10(b) is a quantization matrix corresponding to 16 × 4 orthogonal transform coefficients. Quantization matrix 1002 is obtained by arranging an element at position (row, column) = (2m, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1002.

[0027] In this manner, in this embodiment, the generating unit 103 generates and holds the quantization matrices 901, 902, 1001, and 1002 from the original quantization matrix 800. As a result, regardless of whether the size of the divided rectangular sub-block is 8×16, 16×8, 16×4, or 4×16, a quantization matrix of the corresponding size has already been generated, so that the sub-block can be quantized / dequantized. In particular, the quantization matrices 902 and 1002 in FIG. 9(b) and FIG. 10(b) are generated from the original quantization matrix by enlarging it in one direction and shrinking it in the other direction (thinning out every other row / column). This can reduce the amount of data of the original quantization matrix more than when it is generated by simply shrinking it from the original quantization matrix, and can suppress deterioration of image quality more than when it is generated by simply enlarging it from the original quantization matrix. Therefore, a result with a good balance between rate and distortion can be expected.

[0028] The generating unit 103 then holds the original quantization matrix 800 and the quantization matrices 901, 902, 1001, and 1002 thus generated in a two-dimensional form. It is also possible to hold a plurality of quantization matrices for orthogonal transformation of the same size depending on a prediction method described later, for example, whether intra prediction or inter prediction is used, or whether the encoding target is a luminance block or a chrominance block. In general, since a quantization matrix realizes a quantization process according to the visual characteristics of humans, the elements of the low frequency part corresponding to the upper left part of the quantization matrix are small, and the elements of the high frequency part corresponding to the lower right part are large, as shown in Figs. 8 to 10.

[0029] The quantization matrix encoding unit 113 acquires the quantization matrix stored in a two-dimensional form from the generation unit 103, scans each element to calculate a difference, and generates a one-dimensional matrix (difference matrix) in which the differences are arranged one-dimensionally in the order of calculation. The quantization matrix encoding unit 113 then encodes the generated difference matrix to generate encoded data.

[0030] In this embodiment, since the quantization matrices shown in Figs. 9 and 10 can be generated from the original quantization matrix 800 as described above, the quantization matrix encoding unit 113 encodes only the original quantization matrix 800. Specifically, according to a scanning method in which each element is scanned in the order indicated by the arrow in Fig. 11(a), the difference between each element of the original quantization matrix 800 and the previous element is calculated in the scanning order. For example, the original quantization matrix 800 is scanned by the diagonal scan shown in Fig. 11(a), and after the first element "6" located in the upper left corner, the element "9" located immediately below it is scanned, and the difference "+3" is calculated. In addition, in encoding the first element of the original quantization matrix 800 ("6" in this embodiment), the difference between the first element and a predetermined initial value (for example, "8") is calculated, but this is not limiting, and the difference between any value or the value of the first element itself may be used as the difference.

[0031] In this manner, in this embodiment, the quantization matrix encoding unit 113 generates the difference matrix 1000 shown in Fig. 11(b) from the original quantization matrix 800 using the scanning method in Fig. 11(a). Then, the quantization matrix encoding unit 113 generates encoded data by encoding the difference matrix 1000. In this embodiment, the encoding is performed using the encoding table shown in Fig. 12(a), but the encoding table is not limited to this, and for example, the encoding table shown in Fig. 12(b) may be used.

[0032] In this embodiment, since only the original quantization matrix 800 is coded among the original quantization matrix 800 and the quantization matrices 901, 902, 1001, and 1002, it is possible to reduce the amount of code for a rectangular quantization matrix. In addition, by coding using only one type of scanning method shown in Fig. 11(a) regardless of the shape of the quantization matrix to be generated, it is possible to limit the number of tables for implementing the scanning method of Fig. 11(a) to only one, thereby saving memory.

[0033] The integrated coding unit 111 codes header information necessary for decoding an image, which will be described later, and integrates the header information with the coded data of the quantization matrix generated by the quantization matrix coding unit 113 .

[0034] Next, the coding of the input image will be described. The block division unit 102 divides the input image into a plurality of basic blocks, and outputs each of the divided basic blocks. The prediction unit 104 determines a division method for dividing the basic blocks into a plurality of sub-blocks. An example of the division method for dividing the basic blocks into a plurality of sub-blocks is shown in FIG. 7.

[0035] Fig. 7(b) shows a subblock division method (quadtree division). That is, Fig. 7(b) shows a division method (division pattern) for dividing a basic block 700 into two equal parts horizontally and vertically to divide the basic block 700 into four subblocks (each subblock has a size of 8x8 pixels). On the other hand, Figs. 7(c) to (f) show subblock division methods for dividing the basic block 700 into rectangular subblocks.

[0036] FIG. 7(c) shows a method of dividing a basic block 700 into two equal parts vertically, thereby dividing the basic block 700 into two vertically elongated sub-blocks (each sub-block has a size of 8×16 pixels).

[0037] FIG. 7(d) shows a method of dividing a basic block 700 into two equal parts horizontally, thereby dividing the basic block 700 into two horizontally elongated sub-blocks (each sub-block has a size of 16×8 pixels).

[0038] FIG. 7(e) shows three vertically elongated sub-blocks obtained by dividing the basic block 700 vertically (from the left, a sub-block of 4×16 pixels, a sub-block of 8×16 pixels, and a sub-block of 4×16 pixels) (ternary tree division).

[0039] Figure 7(f) shows three horizontally elongated sub-blocks obtained by dividing the basic block 700 horizontally (a sub-block of 16x4 pixels, a sub-block of 16x8 pixels, and a sub-block of 16x4 pixels from the top) (ternary tree division).

[0040] In this embodiment, one of the division methods (division patterns) shown in Figures 7(b) to (f) is used, but is not limited to Figures 7(b) to (f). As shown in Figure 7(a), the basic block 700 may be divided into sub-blocks without being divided.

[0041] The prediction unit 104 also determines an intra-prediction mode, such as horizontal prediction or vertical prediction, for each sub-block. The prediction unit 104 then divides each basic block into a plurality of sub-blocks according to the determined division method. The prediction unit 104 then generates a predicted image for each sub-block by performing prediction according to the determined intra-prediction mode using an image in the frame memory 108, and obtains a difference between an input image and the predicted image as a prediction error. The prediction unit 104 also generates information indicating a sub-block division method (division pattern), an intra-prediction mode, a motion vector, and other information required for prediction as prediction information.

[0042] The transform / quantization unit 105 generates transform coefficients for each subblock by performing orthogonal transform on the prediction error for each subblock. Then, for each subblock, the transform / quantization unit 105 obtains a quantization matrix corresponding to the size of the subblock from the generation unit 103, and quantizes the transform coefficients of the subblock using the quantization matrix to generate quantized coefficients of the subblock.

[0043] When quantizing the transform coefficients of an 8 pixel by 8 pixel sub-block, the transform / quantization unit 105 obtains the original quantization matrix 800 from the generation unit 103 and quantizes the transform coefficients using the obtained original quantization matrix 800.

[0044] When quantizing the transform coefficients of a sub-block of 8 pixels by 16 pixels, the transform / quantization unit 105 obtains a quantization matrix 901 from the generation unit 103, and quantizes the transform coefficients using the obtained quantization matrix 901.

[0045] When quantizing the transform coefficients of a 16 pixel by 8 pixel sub-block, the transform / quantization unit 105 obtains a quantization matrix 1001 from the generation unit 103 and quantizes the transform coefficients using the obtained quantization matrix 1001.

[0046] When quantizing the transform coefficients of a 4 pixel by 16 pixel sub-block, the transform / quantization unit 105 obtains the quantization matrix 902 from the generation unit 103, and quantizes the transform coefficients using the obtained quantization matrix 902.

[0047] When quantizing the transform coefficients of a 16 pixel by 4 pixel sub-block, the transform / quantization unit 105 obtains a quantization matrix 1002 from the generation unit 103 and quantizes the transform coefficients using the obtained quantization matrix 1002.

[0048] The inverse quantization / inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated by the transform / quantization unit 105 using the quantization matrix used for quantizing the sub-block to generate transform coefficients, and then performs inverse orthogonal transform on the transform coefficients to generate prediction errors.

[0049] The image reproduction unit 107 generates a predicted image from the processed image stored in the frame memory 108 based on the prediction information generated by the prediction unit 104, and reproduces an image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 106. Then, the image reproduction unit 107 stores the reproduced image in the frame memory 108.

[0050] The in-loop filter unit 109 performs in-loop filter processing such as deblocking filtering and sample adaptive offset on the image stored in the frame memory 108 .

[0051] The encoding unit 110 generates encoded data by entropy coding the quantization coefficients (quantized transformation coefficients) generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The entropy coding method is not limited to a specific method, and Golomb coding, arithmetic coding, Huffman coding, etc. can be used.

[0052] The integrated coding unit 111 generates header code data by using the coded data generated by the quantization matrix coding unit 113, and generates a bit stream by multiplexing the coded data generated by the coding unit 110 and the header code data. Then, the integrated coding unit 111 outputs the generated bit stream.

[0053] An example of the bitstream configuration is shown in Fig. 6(a). The sequence header includes code data corresponding to the original quantization matrix 800, and is configured with the encoding results of each element. However, the position at which the data is encoded is not limited to this, and the data may be encoded in a picture header or other header. When changing the quantization matrix within one sequence, the quantization matrix can also be updated by newly encoding it. In this case, all the quantization matrices may be rewritten, or a part of the quantization matrix can be changed by specifying the transform block size of the quantization matrix to be rewritten.

[0054] The encoding process of the input image and quantization matrix by the image encoding device described above will be described with reference to the flowchart in Fig. 3. In step S301, the generation unit 103 generates an original quantization matrix 800, and generates the above quantization matrices 901, 902, 1001, and 1002 from the original quantization matrix 800.

[0055] 11(a) to calculate the difference, and generates a one-dimensional matrix in which the differences are arranged one-dimensionally in the order of calculation as a difference matrix 1000. The quantization matrix encoding unit 113 then encodes the generated difference matrix 1000 to generate encoded data.

[0056] In step S303, the integrated encoding unit 111 encodes header information necessary for encoding an image, and integrates the header information with the encoded data of the quantization matrix generated by the quantization matrix encoding unit 113.

[0057] In step S304, the block dividing unit 102 divides the input image into a plurality of basic blocks.

[0058] In step S305, the prediction unit 104 selects one of the basic blocks divided in step S304 that has not been selected as a selected basic block. The prediction unit 104 then determines a subblock division method (division pattern) and divides the selected basic block into a plurality of subblocks according to the determined subblock division method. The prediction unit 104 also determines an intra-prediction mode for each subblock. The prediction unit 104 then generates a predicted image by performing prediction according to the determined intra-prediction mode using an image in the frame memory 108 for each subblock, and obtains the difference between the input image and the predicted image as a prediction error. The prediction unit 104 also generates information necessary for prediction, such as information indicating the subblock division method, an intra-prediction mode, and a motion vector, as prediction information.

[0059] In step S306, the transform / quantization unit 105 generates transform coefficients for each sub-block by performing orthogonal transform on the prediction error for each sub-block. Then, for each sub-block, the transform / quantization unit 105 quantizes the transform coefficients of the sub-block using a quantization matrix corresponding to the size of the sub-block from among the quantization matrices generated in step S301, to generate quantized coefficients for the sub-block.

[0060] In step S307, the inverse quantization and inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated in step S306 using the quantization matrix used in the quantization of the sub-block to generate transform coefficients, and then performs inverse orthogonal transform on the generated transform coefficients to generate prediction errors.

[0061] In step S308, the image reproduction unit 107 generates a predicted image from the processed image stored in the frame memory 108, based on the prediction information generated by the prediction unit 104. Then, the image reproduction unit 107 reproduces an image from the predicted image and the prediction error generated by the inverse quantization / inverse transform unit 106. Then, the image reproduction unit 107 stores the reproduced image in the frame memory 108.

[0062] In step S309, the encoding unit 110 generates encoded data by entropy encoding the quantization coefficients generated by the transform / quantization unit 105 and the prediction information generated by the prediction unit 104. The integrated encoding unit 111 generates header encoded data by using the encoded data generated by the quantization matrix encoding unit 113, and multiplexes the encoded data generated by the encoding unit 110 and the header encoded data to generate a bit stream.

[0063] In step S310, the control unit 199 determines whether or not all basic blocks have been coded. If the result of this determination is that all basic blocks have been coded, the process proceeds to step S311, and if there are basic blocks remaining that have not yet been coded, the process proceeds to step S305.

[0064] In step S311, the in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the image stored in the frame memory .

[0065] According to the present embodiment described above, in particular, a plurality of quantization matrices are generated from one quantization matrix in step S301, and only one quantization matrix is ​​encoded in step S302, thereby reducing the amount of code for the quantization matrix. As a result, the amount of data in the entire bit stream generated is reduced, and the compression efficiency can be improved. In particular, the quantization matrices in FIG. 9(b) and FIG. 10(b) are generated by enlarging the original quantization matrix in one direction and shrinking the original quantization matrix in the other direction. This can reduce the amount of data for the original quantization matrix more than when it is generated by simply shrinking the original quantization matrix, and can suppress deterioration of image quality more than when it is generated by simply enlarging the original quantization matrix. Therefore, a result with a good balance between rate and distortion can be expected.

[0066] In this embodiment, multiple quantization matrices are generated from one quantization matrix. However, it is also possible to select whether to generate each quantization matrix from another quantization matrix or to encode each element, and to encode an identifier indicating the selection result in the header. For example, it is also possible to encode information indicating whether each element of the quantization matrix is ​​generated from another quantization matrix or encoded individually in the header as a quantization matrix encoding method information code, and generate a bit stream as shown in FIG. 6(b). This makes it possible to selectively generate a bit stream that prioritizes image quality control according to the size of the subblock and a bit stream with a smaller amount of code for the quantization matrix.

[0067] Also, in this embodiment, a method has been described for generating quantization matrices 901, 902, 1001, and 1002 from original quantization matrix 800. However, a method for generating rectangular quantization matrices from a square quantization matrix is ​​not limited to the method described in the first embodiment.

[0068] For example, a method of generating rectangular quantization matrices 1201 to 1204 shown in FIGS. 13(a) and 13(b) and FIGS. 14(a) and 14(b), respectively, from the original quantization matrix 800 may be employed.

[0069] Quantization matrix 1201 shown in Fig. 13(a) is a quantization matrix corresponding to 8 x 16 orthogonal transform coefficients. Each row of original quantization matrix 800 is arranged in the even rows of quantization matrix 1201. In the odd rows of quantization matrix 1201, elements interpolated from the even rows on either side of the odd row are arranged. For example, element "14" at position (row, column) = (1, 2) in quantization matrix 1201 is determined as a value interpolated from element "13" in the even row immediately above it and element "15" in the even row immediately below it.

[0070] Quantization matrix 1202 shown in Fig. 13(b) is a quantization matrix corresponding to 4 × 16 orthogonal transform coefficients. Quantization matrix 1202 is made up of only even columns, excluding odd columns, of quantization matrix 1201 generated from original quantization matrix 800 as described above.

[0071] Quantization matrix 1203 shown in Fig. 14(a) is a quantization matrix corresponding to 16 x 8 orthogonal transform coefficients. Each column of original quantization matrix 800 is arranged in an even-numbered column of quantization matrix 1203. In odd-numbered columns of quantization matrix 1203, elements interpolated from even-numbered columns on either side of the odd-numbered column are arranged. For example, element "14" at position (row, column) = (2, 1) in quantization matrix 1203 is determined as a value interpolated from element "13" in the adjacent column to the left and element "15" in the adjacent column to the right.

[0072] Quantization matrix 1204 shown in Fig. 14(b) is a quantization matrix corresponding to 16 × 4 orthogonal transform coefficients. Quantization matrix 1204 is made up of only the even rows of quantization matrix 1203 generated from original quantization matrix 800 as described above, excluding the odd rows.

[0073] In this way, another quantization matrix is ​​generated by expanding or contracting the original quantization matrix 800 in the row or column direction. This generation method allows for more detailed quantization control according to frequency components, resulting in improved image quality.

[0074] Furthermore, in this embodiment, elements in even-numbered columns and even-numbered rows of the original quantization matrix 800 are used when generating the quantization matrix 902 and the quantization matrix 1002. However, elements in odd-numbered columns and odd-numbered rows of the original quantization matrix 800 may be used to generate the quantization matrix.

[0075] For example, a quantization matrix 1601 corresponding to 4×16 orthogonal transform coefficients as shown in FIG. 15(a), or a quantization matrix 1602 corresponding to 16×4 orthogonal transform coefficients as shown in FIG. 15(b) may be generated.

[0076] Quantization matrix 1601 is obtained by placing the element at position (row, column) = (m, 2n+1) in original quantization matrix 800 at position (row, column) = (2m, n) and position (row, column) = (2m+1, n) in quantization matrix 1601.

[0077] Quantization matrix 1602 is obtained by placing the element at position (row, column) = (2m+1, n) in original quantization matrix 800 at position (row, column) = (m, 2n) and position (row, column) = (m, 2n+1) in quantization matrix 1602.

[0078] In this way, it is possible to generate multiple quantization matrices with the same shape but different elements from one original quantization matrix. Flags indicating the selection of even or odd columns of the original quantization matrix and flags indicating the selection of even or odd rows may be coded.

[0079] Furthermore, a configuration may be adopted in which multiple quantization matrices of the same shape thus generated are switched according to the type of orthogonal transform method used. For example, when the orthogonal transform is based on discrete cosine transform, quantization matrix 902 or quantization matrix 1002 is generated, and when the orthogonal transform is based on discrete sine transform, quantization matrix 1601 or quantization matrix 1602 is generated. This makes it possible to use a quantization matrix according to the properties of the orthogonal transform used.

[0080] Furthermore, if there is a limit to the size of the orthogonal transform used, the generation and encoding of some of the quantization matrices may be omitted. For example, if the orthogonal transform is not performed on the pixels of a subblock of 4×16 size or 16×4 size, the generation of the quantization matrix 902 and the quantization matrix 1002 may be omitted. Furthermore, if the orthogonal transform is not performed on the pixels of a subblock of any of the sizes of 8×8, 16×8, 8×16, 16×4, and 4×16, the encoding of the original quantization matrix 800 itself may be omitted. In that case, the generation of the quantization matrices 901, 902, 1001, and 1002 is naturally omitted. However, even if the orthogonal transform is not used on the pixels of the 8×8 subblock, if the orthogonal transform is performed on the pixels of at least one of the sizes of 16×8, 8×16, 16×4, and 4×16, the original quantization matrix 800 is encoded and a quantization matrix corresponding to the orthogonal transform is generated. This allows only the quantization matrices that are used to be coded and generated, thereby reducing the amount of unnecessary code when the quantization matrices are not used.

[0081] [Second embodiment] In this embodiment, an image decoding device that decodes an input image encoded by the image encoding device according to the first embodiment will be described. Differences from the first embodiment will be described below, and unless otherwise noted below, the present embodiment is the same as the first embodiment. An example of the functional configuration of the image decoding device according to this embodiment will be described with reference to the block diagram of FIG.

[0082] The control unit 299 controls the operation of the entire image decoding device. The separate decoding unit 202 obtains a bit stream generated by the image coding device, separates information related to the decoding process and coded data related to coefficients from the bit stream, and decodes the coded data present in the header of the bit stream. In this embodiment, the separate decoding unit 202 separates coded data of the quantization matrix from the bit stream. That is, the separate decoding unit 202 performs the reverse operation of the integrated coding unit 111 described above.

[0083] The quantization matrix decoding unit 209 obtains the coded data of the quantization matrix separated from the bit stream by the separation decoding unit 202, and generates a quantization matrix by decoding the obtained coded data.

[0084] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the coded data separated from the bit stream by the separation decoding unit 202. The inverse quantization and inverse transform unit 204 performs the same operation as the inverse quantization and inverse transform unit 106 of the above-mentioned image coding device. The inverse quantization and inverse transform unit 204 obtains transform coefficients by performing inverse quantization on the quantization coefficients based on the quantization matrix decoded by the quantization matrix decoding unit 209, and obtains prediction errors by performing inverse orthogonal transform on the transform coefficients.

[0085] The image reproduction unit 205 generates a predicted image by referring to an image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. The image reproduction unit 205 then generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization and inverse transform unit 204, and stores the generated reproduced image in the frame memory 206.

[0086] The in-loop filter unit 207 performs in-loop filter processing such as a deblocking filter on the reconstructed image stored in the frame memory 206. The reconstructed image stored in the frame memory 206 is output as appropriate by the control unit 299. The output destination of the reconstructed image is not limited to a specific output destination, and for example, the reconstructed image may be displayed on a display screen of a display device such as a display, or the reconstructed image may be output to a projection device such as a projector.

[0087] Next, the operation of the image decoding device having the above configuration (decoding process of bit stream) will be described. In this embodiment, the bit stream input to the separation decoding unit 202 is a bit stream for each frame of a moving image, but it may be a bit stream of a still image. In addition, in this embodiment, for ease of explanation, only intra prediction decoding process will be described, but it is not limited to this and can also be applied to inter prediction decoding process.

[0088] The separate decoding unit 202 obtains a bit stream for one frame generated by the image encoding device, separates encoded data related to information on the decoding process and coefficients from the bit stream, and decodes the encoded data present in the header of the bit stream. The separate decoding unit 202 extracts quantization matrix code data (quantization matrix data) from the sequence header of the bit stream in Fig. 6(a), and sends the quantization matrix code data to the quantization matrix decoding unit 209. The separate decoding unit 202 also outputs encoded data of picture data in units of basic blocks to the decoding unit 203.

[0089] The quantization matrix decoding unit 209 generates a one-dimensional difference matrix by decoding the coded data of the quantization matrix separated from the bit stream by the separation decoding unit 202. The coding table used on the coding side (in the first embodiment, the coding table used on the coding side among the coding tables of FIG. 12(a) and FIG. 12(b)) is used to decode the coded data of the quantization matrix. The quantization matrix decoding unit 209 then generates a two-dimensional quantization matrix (original quantization matrix) by inversely scanning the generated one-dimensional difference matrix. This process is the reverse of the process of the quantization matrix coding unit 113 described in the first embodiment. That is, in this embodiment, the difference matrix 1000 in FIG. 11(b) generates the original quantization matrix 800 in FIG. 8 using the scanning method of FIG. 11(a). The quantization matrix decoding unit 209 then generates quantization matrices 901, 902, 1001, and 1002 from the generated original quantization matrix 800 in the same manner as in the first embodiment. At this time, the flags indicating the selection of even or odd columns and the flags indicating the selection of even or odd rows of the original quantization matrix are decoded as necessary. Then, generation from the original quantization matrix is ​​performed according to these flags.

[0090] The decoding unit 203 obtains quantization coefficients and prediction information by decoding the coded data separated from the bit stream by the separation decoding unit 202. The inverse quantization and inverse transform unit 204 selects a quantization matrix according to the size of the sub-block to be decoded from among the original quantization matrix 800 and the quantization matrices 901, 902, 1001, and 1002 generated by the quantization matrix decoding unit 209, based on information indicating a division pattern of the sub-block, which is one of the prediction information. The inverse quantization and inverse transform unit 204 then obtains transform coefficients by performing inverse quantization on the quantization coefficients of the sub-block to be decoded using the selected quantization matrix, and obtains prediction errors by performing inverse orthogonal transform on the transform coefficients.

[0091] The image reproduction unit 205 generates a predicted image by referring to an image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. The image reproduction unit 205 then generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization and inverse transform unit 204, and stores the generated reproduced image in the frame memory 206. The in-loop filter unit 207 performs in-loop filtering such as a deblocking filter on the reproduced image stored in the frame memory 206.

[0092] The decoding process of one frame of a bit stream by the image decoding device described above will be described with reference to the flowchart in Fig. 4. In step S401, the separate decoding unit 202 obtains one frame of a bit stream generated by the image coding device. The separate decoding unit 202 then separates information related to the decoding process and coded data related to coefficients from the bit stream, and decodes the coded data present in the header of the bit stream.

[0093] In step S402, the quantization matrix decoding unit 209 generates a one-dimensional difference matrix by decoding the encoded data of the quantization matrix separated from the bit stream by the separation decoding unit 202. The quantization matrix decoding unit 209 then inversely scans the generated one-dimensional difference matrix to generate a two-dimensional original quantization matrix 800. The quantization matrix decoding unit 209 then generates quantization matrices 901, 902, 1001, and 1002 from the generated original quantization matrix 800 in the same manner as in the first embodiment. In particular, the quantization matrices 902 and 1002 in FIG. 9(b) and FIG. 10(b) are generated by enlarging the original quantization matrix in one direction and shrinking the original quantization matrix in the other direction. This can reduce the amount of data of the original quantization matrix compared to when it is generated by simply shrinking the original quantization matrix, and can suppress deterioration of image quality compared to when it is generated by simply enlarging the original quantization matrix. Therefore, a result with a good balance between rate and distortion can be expected.

[0094] Note that when the sizes of the quantization matrices 902 and 1002 in FIGS. 9(b) and 10(b) are PXQ, which is the size of the quantization matrix generated when the size of the original quantization matrix is NXN, P and Q are positive integers that satisfy P < N < Q or Q < N < P. Note that when dividing a block into sub-blocks, it may be adaptively generated according to the division information indicating which type of pattern among FIGS. 7(b) to 7(f) is used.

[0095] The processes of steps S403 to S406 are performed for each basic block in the input image. In step S403, the decoding unit 203 obtains quantization coefficients and prediction information by decoding the encoded data separated from the bit stream by the separation decoding unit 202.

[0096] In step S404, the inverse quantization / inverse transformation unit 204 selects a quantization matrix corresponding to the size and shape of the sub-block to be decoded from among the quantization matrices generated by the quantization matrix decoding unit 209. Then, the inverse quantization / inverse transformation unit 204 obtains transformation coefficients by performing inverse quantization on the quantization coefficients of the sub-block to be decoded using the selected quantization matrix, and obtains a prediction error by performing an inverse orthogonal transformation on the transformation coefficients.

[0097] In step S405, the image reproduction unit 205 generates a predicted image by referring to the image stored in the frame memory 206 based on the prediction information decoded by the decoding unit 203. Then, the image reproduction unit 205 generates a reproduced image using the generated predicted image and the prediction error obtained by the inverse quantization / inverse transformation unit 204, and stores the generated reproduced image in the frame memory 206.

[0098] In step S406, the control unit 299 judges whether or not the decoding of all basic blocks included in the bit stream is completed. If the result of this judgment is that the decoding of all basic blocks included in the bit stream is completed, the process proceeds to step S407. On the other hand, if there are sub-blocks that have not yet been decoded among all basic blocks included in the bit stream, the process from step S403 onwards is repeated for the sub-blocks that have not yet been decoded.

[0099] In step S407, the in-loop filter unit 207 performs in-loop filtering, such as deblocking filtering, on the reconstructed image stored in the frame memory 206.

[0100] According to the present embodiment described above, since only the original quantization matrix generated in the first embodiment is encoded, it is possible to decode a bitstream with a reduced amount of code of the quantization matrix. Also, since the element scanning method used for decoding the quantization matrix is ​​limited to one type shown in Fig. 11(a), it is possible to limit the table for implementing the scanning method shown in Fig. 11(a) to only one, thereby saving memory.

[0101] In this embodiment, multiple quantization matrices are generated from one quantization matrix. However, an identifier for selecting whether to generate each quantization matrix by deriving it from another quantization matrix or to decode each element coded independently of another quantization matrix may be decoded from a header. For example, information indicating whether to generate each element of a quantization matrix from another quantization matrix or to decode it individually may be decoded as a quantization matrix coding method information code from the header of the bit stream in FIG. 6(b). This allows a bit stream to be decoded in which a priority is given to image quality control according to the size of the subblock or the amount of code of the quantization matrix is ​​reduced.

[0102] In the present embodiment, a method for generating quantization matrices 901, 902, 1001, and 1002 from an original quantization matrix 800 has been described. However, as with the image encoding device side, a method for generating a rectangular quantization matrix from a square quantization matrix is ​​not limited to the method described in the first embodiment. For example, a method for generating quantization matrices 1201 to 1204 shown in Figs. 13(a) and 13(b) and Figs. 14(a) and 14(b), respectively, from an original quantization matrix 800 may be adopted. This allows finer quantization control according to frequency components to be performed, resulting in a decoded bitstream with improved image quality.

[0103] In addition, when a certain size of quantization matrix is ​​not used due to restrictions on the size of the orthogonal transform used, the decoding and generation of the unused quantization matrix may be omitted. For example, when an orthogonal transform is not performed on a subblock of 4×16 or 16×4 pixel size, the generation of the quantization matrix 902 or 1002 may be omitted. Furthermore, when an orthogonal transform is not performed on a subblock of 8×8, 16×8, 8×16, 16×4, or 4×16 pixel size, the decoding of the original quantization matrix 800 itself may be omitted. In that case, the generation of the quantization matrices 901, 902, 1001, and 1002 is naturally omitted. However, even when an orthogonal transform is not performed on a subblock of 8×8 pixel size, when an orthogonal transform is performed on at least one of the rectangular subblocks of 16×8, 8×16, 16×4, or 4×16 size, the original quantization matrix 800 is decoded and a quantization matrix corresponding to the orthogonal transform is generated. This allows only the quantization matrices to be used to be decoded and generated, making it possible to decode a bitstream with a reduced amount of unnecessary coding when no quantization matrix is ​​used.

[0104] In this embodiment, the image decoding device has been described as being separate from the image encoding device according to the first embodiment, but the image decoding device and the image encoding device according to the first embodiment may be integrated into one device. In that case, this device can encode an input image and decode the encoded input image as necessary.

[0105] [Third embodiment] All of the functional units shown in Figures 1 and 2 may be implemented in hardware, or some of them may be implemented in software (computer programs). In that case, a computer device having frame memory 108 or frame memory 206 as a memory device and capable of executing the computer programs can be applied to the above image encoding device or image decoding device. An example of the hardware configuration of a computer device applicable to the above image encoding device or image decoding device will be described with reference to the block diagram of Figure 5.

[0106] The CPU 501 executes various processes using computer programs and data stored in the RAM 502 and the ROM 503. As a result, the CPU 501 controls the operation of the entire computer device, and executes or controls each of the processes described above as being performed by the image encoding device and image decoding device.

[0107] The RAM 502 has areas for storing computer programs and data loaded from the ROM 503 or the external storage device 506, and data received from the outside via an I / F (interface) 507 (for example, the above-mentioned moving image and still image data). The RAM 502 further has a work area used when the CPU 501 executes various processes. In this way, the RAM 502 can provide various areas as appropriate. The ROM 503 stores setting data, startup programs, and the like for the computer device.

[0108] The operation unit 504 is a user interface such as a keyboard, a mouse, or a touch panel, and the user can input various instructions to the CPU 501 by operating it.

[0109] The display unit 505 is configured with a liquid crystal screen, a touch panel screen, or the like, and can display the processing results by the CPU 501 as images, characters, and the like. For example, a reproduced image decoded by the above-mentioned image decoding device may be displayed on this display unit 505. Note that the display unit 505 may be a projection device such as a projector that projects images and characters.

[0110] The external storage device 506 is a large-capacity information storage device such as a hard disk drive device. The external storage device 506 stores an OS (operating system), computer programs and data for causing the CPU 501 to execute or control the various processes described above as those performed by the image encoding device and image decoding device. The computer programs stored in the external storage device 506 include computer programs for causing the CPU 501 to realize the functions of each functional unit other than the frame memory 108 and the frame memory 206. The data stored in the external storage device 506 also includes various information required for encoding and decoding, such as the encoding tables shown in Figures 12(a) and 12(b).

[0111] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501, and are processed by the CPU 501. The frame memory 108 and the frame memory 206 can be implemented by memory devices such as the RAM 502, the ROM 503, and the external storage device 506.

[0112] The I / F 507 functions as an interface for performing data communication with external devices. For example, moving images and still images can be acquired from an external server device or imaging device to the RAM 502 or the external storage device 506 via the I / F 507.

[0113] The CPU 501, RAM 502, ROM 503, operation unit 504, display unit 505, external storage device 506, and I / F 507 are all connected to a bus 508. Note that the configuration shown in Fig. 5 is merely an example of a hardware configuration of a computer device applicable to the above-mentioned image encoding device and image decoding device, and various changes / modifications are possible.

[0114] [Fourth embodiment] The numerical values ​​used in the above description are used for the purpose of concrete description, and it is not intended that the above embodiments are limited to the numerical values ​​used. For example, the size of the quantization matrix and each element of the quantization matrix are not limited to the numerical values ​​described above.

[0115] In addition, a part or all of the embodiments described above may be used in appropriate combination. In addition, a part or all of the embodiments described above may be used selectively.

[0116] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0117] 102: Block division unit 103: Generation unit 104: Prediction unit 105: Transformation and quantization unit 106: Inverse quantization and inverse transformation unit 107: Image reproduction unit 108: Frame memory 109: In-loop filter unit 110: Encoding unit 111: Integrated encoding unit 113: Quantization matrix encoding unit

Claims

1. An image encoding device that encodes an image, comprising: A generating means for generating a second quantization matrix having a size of 4×16 (the size of 4×16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; a quantization means for quantizing transform coefficients in a sub-block of a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); The generating means includes at least (a) directly reducing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction when generating the second quantization matrix from the first quantization matrix, by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in a vertical direction by arranging an element in m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element in 2m rows and n columns of the second quantization matrix and an element in (2m+1) rows and n columns of the second quantization matrix; generating the second quantization matrix; The generating means is capable of generating a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix.

1. An image encoding device comprising:

2. Furthermore, encoding means for encoding the first quantization matrix; means for outputting a bit stream including the encoded data of the image and the encoded data of the first quantization matrix; The image encoding device according to claim 1 , further comprising:

3. In a case where a transform process is performed on a sub-block having a size corresponding to the 4×16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​executable, 2. The image encoding device according to claim 1, characterized in that when a conversion process is not performed on a sub-block of a size corresponding to the 4 x 16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​not performed.

4. 2. The image encoding device according to claim 1, wherein the sub-block having a size corresponding to the 4x16 size corresponds to the first sub-block or the third sub-block among three sub-blocks obtained by vertically dividing a block having a size of 16x16 into thirds, the first sub-block having a size of 4x16, a second sub-block having a size of 8x16, and a third sub-block having a size of 4x16.

5. Furthermore, The generating means includes: generating a fourth quantization matrix having a size of 16×4 (the size of 16×4 indicates a size of 16 in the horizontal direction and a size of 4 in the vertical direction) from the first quantization matrix having a size of 8×8; the fourth quantization matrix has an sth row and a tth column, where s is an integer satisfying 0≦s≦3 and t is an integer satisfying 0≦t≦15; The generating means includes at least (a) directly reducing the elements of the first quantization matrix having a size of 8×8 in the vertical direction when generating the fourth quantization matrix from the first quantization matrix, by excluding all elements of the first row, the third row, the fifth row, and the seventh row of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8×8 in the vertical direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the horizontal direction by arranging an element of 2m rows and n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element of m rows and 2n columns in the fourth quantization matrix and an element of m rows and (2n+1) columns in the fourth quantization matrix; generating the fourth quantization matrix; The quantization means quantizes the transform coefficients in the sub-blocks having a size corresponding to the 16×4 size using the fourth quantization matrix.

2. The image encoding device according to claim 1 .

6. The image encoding device described in claim 5, characterized in that when conversion processing is not performed on sub-blocks of a size corresponding to the 4 x 16 size and sub-blocks of a size corresponding to the 16 x 4 size, the process of generating the second quantization matrix from the first quantization matrix and the process of generating the fourth quantization matrix from the first quantization matrix are not executed.

7. 6. The image encoding device according to claim 5, wherein the sub-block of a size corresponding to the 16x4 size corresponds to the first sub-block or the third sub-block of three sub-blocks obtained by horizontally dividing a block of 16x16 size into thirds, the first sub-block of 16x4 size, the second sub-block of 16x8 size, and the third sub-block of 16x4 size.

8. The encoding means includes at least (a) encoding a difference value between the element in the 0th row and 0th column of the first quantization matrix and a predetermined value; (b) encoding a difference value between an element in the 1st row, 0th column of the first quantization matrix, which is an element next to the element in the 0th row, 0th column in a predetermined scanning order, and the element in the 0th row, 0th column of the first quantization matrix; (c) encoding a difference value between an element in the 0th row and the 1st column of the first quantization matrix, which is an element next to the 1st row and the 0th column in the predetermined scanning order, and the element in the 1st row and the 0th column of the first quantization matrix; (d) encoding a difference value between an element in the second row and the 0th column of the first quantization matrix, which is the element next to the element in the 0th row and the 1st column in the predetermined scanning order, and an element in the 0th row and the 1st column of the first quantization matrix; Encoding the first quantization matrix 3. The image encoding device according to claim 2.

9. An image decoding device that decodes an encoded image, comprising: A generating means for generating a second quantization matrix having a size of 4×16 (the size of 4×16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; a dequantization means for dequantizing the quantized transform coefficients in the sub-blocks having a size corresponding to the size of 4×16 using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); The generating means includes at least (a) directly reducing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction when generating the second quantization matrix from the first quantization matrix, by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in a vertical direction by arranging an element in m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element in 2m rows and n columns of the second quantization matrix and an element in (2m+1) rows and n columns of the second quantization matrix; generating the second quantization matrix; The generating means is capable of generating a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) from the first quantization matrix.

2. An image decoding device comprising:

10. In a case where a transform process is performed on a sub-block having a size corresponding to the 4×16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​executable, 10. The image decoding device according to claim 9, characterized in that when a transformation process is not performed on a sub-block of a size corresponding to the 4x16 size, a process of generating the second quantization matrix from the first quantization matrix is ​​not performed.

11. 10. The image decoding device according to claim 9, wherein the sub-block having a size corresponding to the 4x16 size corresponds to the first sub-block or the third sub-block among three sub-blocks including a first sub-block having a size of 4x16, a second sub-block having a size of 8x16, and a third sub-block having a size of 4x16 obtained by vertically dividing a block having a size of 16x16 into thirds.

12. Furthermore, The generating means includes: Generate a fourth quantization matrix having a size of 16×4 from the first quantization matrix having a size of 8×8 (the size of 16×4 indicates a size of 16 in the horizontal direction and a size of 4 in the vertical direction); the fourth quantization matrix has an sth row and a tth column, where s is an integer satisfying 0≦s≦3 and t is an integer satisfying 0≦t≦15; The generating means includes at least (a) directly reducing the elements of the first quantization matrix having a size of 8×8 in the vertical direction when generating the fourth quantization matrix from the first quantization matrix, by excluding all elements of the first row, the third row, the fifth row, and the seventh row of the first quantization matrix without increasing the elements of the first quantization matrix having a size of 8×8 in the vertical direction; and (b) increasing the elements of the first quantization matrix having a size of 8×8 in the horizontal direction by arranging an element of 2m rows and n columns (m and n are integers equal to or greater than 0) in the first quantization matrix as an element of m rows and 2n columns in the fourth quantization matrix and an element of m rows and (2n+1) columns in the fourth quantization matrix; generating the fourth quantization matrix; The inverse quantization means inversely quantizes the quantized transform coefficients in the sub-blocks having a size corresponding to the 16×4 size using the third quantization matrix.

10. The image decoding device according to claim 9.

13. The image decoding device described in claim 12, characterized in that when conversion processing is not performed on sub-blocks of a size corresponding to the 4x16 size and sub-blocks of a size corresponding to the 16x4 size, the process of generating the second quantization matrix from the first quantization matrix and the process of generating the third quantization matrix from the first quantization matrix are not executed.

14. 13. The image decoding device according to claim 12, wherein the sub-block of a size corresponding to the 16x4 size corresponds to the first or third sub-block of three sub-blocks obtained by horizontally dividing a block of 16x16 size into thirds, the first or third sub-block being the 16x4 sub-block, the second or third sub-block being the 16x8 sub-block, and the third sub-block being the 16x4 sub-block.

15. Furthermore, at least (a) deriving a one-dimensional matrix obtained from a plurality of difference values ​​including a difference value between an element in the 0th row and 0th column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in a predetermined scanning order, by decoding encoded data corresponding to the first quantization matrix; (b) deriving an element in the 0th row and 0th column of the first quantization matrix using a first element, the first element being a first element in the one-dimensional matrix; (c) deriving an element in the first row and the 0th column of the first quantization matrix, which is an element in the first row and the 0th column of the first quantization matrix, which is an element in the first row and the 0th column of the first quantization matrix, in a predetermined scanning order, using a second element that is an element subsequent to the first element in the one-dimensional matrix; (d) deriving an element in the 0th row and the 1st column of the first quantization matrix, which is an element subsequent to the element in the 1st row and the 0th column in the predetermined scanning order, using a third element that is an element subsequent to the second element in the one-dimensional matrix; (e) deriving an element in the second row and the zeroth column of the first quantization matrix, which is an element next to the element in the zeroth row and the first column in the predetermined scanning order, by using a fourth element that is an element next to the third element in the one-dimensional matrix; A decoding means for decoding the first quantization matrix is ​​provided.

10. The image decoding device according to claim 9.

16. 1. An image encoding method for encoding an image, comprising the steps of: A generating step of generating a second quantization matrix having a size of 4×16 (the size of 4×16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; a quantization step of quantizing transform coefficients in a sub-block of a size corresponding to the 4×16 size using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); In the generating step, at least (a) directly reducing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction when generating the second quantization matrix from the first quantization matrix, by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction; and (b) generating the second quantization matrix by vertically increasing the elements of the first quantization matrix having a size of 8×8 such that an element in m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix is ​​arranged as an element in 2m rows and n columns of the second quantization matrix and as an element in (2m+1) rows and n columns of the second quantization matrix; In the generating step, a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) can be generated from the first quantization matrix.

13. An image coding method comprising:

17. 1. An image decoding method for decoding an encoded image, comprising the steps of: A generating step of generating a second quantization matrix having a size of 4×16 (the size of 4×16 indicates a size of 4 in the horizontal direction and a size of 16 in the vertical direction) from a first quantization matrix having a size of 8×8; a dequantization step of dequantizing the quantized transform coefficients in the sub-blocks of a size corresponding to the size of 4×16 using the second quantization matrix; the first quantization matrix has an r-th row and an r-th column (r is an integer satisfying 0≦r≦7), the second quantization matrix has a p-th row and a q-th column (p is an integer satisfying 0≦p≦15, and q is an integer satisfying 0≦q≦3); In the generating step, at least (a) directly reducing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction when generating the second quantization matrix from the first quantization matrix, by excluding all elements in the first, third, fifth, and seventh columns of the first quantization matrix without increasing the elements of the first quantization matrix having the size of 8×8 in the horizontal direction; and (b) generating the second quantization matrix by vertically increasing the elements of the first quantization matrix having a size of 8×8 such that an element in m rows and 2n columns (m and n are integers equal to or greater than 0) in the first quantization matrix is ​​arranged as an element in 2m rows and n columns of the second quantization matrix and as an element in (2m+1) rows and n columns of the second quantization matrix; In the generating step, a third quantization matrix having a size of 8×16 (the size of 8×16 indicates a size of 8 in the horizontal direction and a size of 16 in the vertical direction) can be generated from the first quantization matrix.

2. An image decoding method comprising:

18. A computer program for causing a computer to function as each of the means of the image coding apparatus according to any one of claims 1 to 8.

19. A computer program for causing a computer to function as each of the means of the image decoding device according to any one of claims 9 to 15.

Images