Image decoding device, image encoding device, image decoding and encoding methods, and program

Abstract

To provide a method and a device that improve the efficiency in encoding image data by reducing the amount of encoding in a quantization matrix.SOLUTION: An image encoding device has a quantization unit that quantizes a block prediction error by using a quantization matrix, and an encoding unit that encodes the quantization matrix. The image encoding device has: a first mode for encoding information indicating that a quantization matrix to be encoded is the same as an already-encoded quantization matrix and not encoding elements of the quantization matrix to be encoded; a second mode for encoding information indicating encoding the differences between the elements of the quantization matrix to be encoded and elements of the already-encoded quantization matrix and encoding the differences between the elements of the quantization matrix to be encoded and the elements of the already-encoded quantization matrix; and a third mode for encoding information indicating encoding the differences between the elements in the quantization matrix to be encoded and encoding the differences between the elements in the quantization matrix to be encoded.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to image encoding and decoding processes. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter simply referred to as HEVC) is known as a coding method for compressing and recording moving images. To improve coding efficiency, HEVC uses basic blocks larger than conventional macroblocks (16x16 pixels). These large basic blocks are called Coding Tree Units (CTUs) and can be up to 64x64 pixels in size. CTUs are further divided into sub-blocks, which serve as units for prediction and transformation.

[0003] Furthermore, HEVC uses a process called a quantization matrix, which weights coefficients after orthogonal transformation (hereinafter referred to as orthogonal transform coefficients) according to frequency components. By further reducing data of high-frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.

[0004] In recent years, efforts have begun to internationally standardize a more efficient coding method as a successor to HEVC. The Joint Video Experts Team (JVET) was established between ISO / IEC and ITU-T, and standardization is underway as the Versatile Video Coding (VVC) coding method (hereinafter referred to as VVC). To improve coding efficiency, the maximum basic block size in VVC is 128x128 pixels, and rectangular sub-block division is also being considered in addition to the conventional square sub-block division. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-38758 Summary of the Invention [Problem to be solved by the invention]

[0006] As with HEVC, the introduction of quantization matrices is being considered for VVC. Furthermore, more types of subblock divisions than HEVC, including rectangular shapes, are being considered for VVC. Because the distribution of orthogonal transform coefficients corresponding to each subblock division differs depending on the size and shape of the orthogonal transform, it is desirable to define an optimal quantization matrix according to the size and shape of the subblock. However, defining individual quantization matrices for every subblock shape would unnecessarily increase the amount of code for the quantization matrices.

[0007] The present invention has been made in view of the above problem, and aims to provide a technique for reducing the amount of code in a quantization matrix. [Means for solving the problem]

[0008] In order to solve this problem, for example, an image decoding device of the present invention has the following configuration: decoding means for decoding the first quantization matrix; and an inverse quantization means for inversely quantizing residual coefficients using the first quantization matrix; the decoding means decodes first information representing a mode of decoding the first quantization matrix from a scaling_list_data syntax structure; When the decoded first information indicates that a first mode is used, the decoding means decodes second information for identifying a second quantization matrix to be referenced from the scaling_list_data syntax structure, and derives the first quantization matrix by determining that the second quantization matrix identified based on the second information is identical to the first quantization matrix; If the decoded first information indicates that a second mode is used, the decoding means decodes the second information for identifying the second quantization matrix from the scaling_list_data syntax structure; (a) associating a first element in a first one-dimensional matrix consisting of a plurality of elements, the first element being an element of a difference value between the element in the first row and the first column of the first quantization matrix and the element in the first row and the first column of the second quantization matrix specified based on the second information, with the first row and the first column of a two-dimensional matrix as a first element in a predetermined scanning order, and deriving the element in the first row and the first column of the first quantization matrix using the first element associated with the first row and the first column of the second quantization matrix; (b) associating a second element in the first one-dimensional matrix, the second element being a difference value element between the element in the second row, the first column of the first quantization matrix and the element in the second row, the first column of the second quantization matrix, with the second row, the first column of the two-dimensional matrix as a second element in the predetermined scanning order, and deriving the element in the second row, the first column of the first quantization matrix using the second element associated with the second row, the first column and the element in the second row, the first column of the second quantization matrix; (c) associating a third element in the first one-dimensional matrix, the third element being a difference value element between the element in the first row, second column of the first quantization matrix and the element in the first row, second column of the second quantization matrix, with the first row, second column of the two-dimensional matrix as a third element in the predetermined scanning order, and decoding the first quantization matrix by using the third element associated with the first row, second column and the element in the first row, second column of the second quantization matrix to derive the element in the first row, second column of the first quantization matrix; When the decoded first information indicates that a third mode is used, the decoding means decodes, from the scaling_list_data syntax structure, a plurality of difference values including a difference value between an element in the first row and first column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order; (a) using a first element in a second one-dimensional matrix obtained from the plurality of difference values, deriving an element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order; (b) using a second element in the second one-dimensional matrix, deriving an element in the second row and first column of the first quantization matrix corresponding to the second element in the predetermined scanning order; (c) using a third element in the second one-dimensional matrix, deriving an element in the first row and second column of the first quantization matrix that corresponds to the third element in the predetermined scanning order; (d) decoding the first quantization matrix by using a fourth element in the second one-dimensional matrix to derive an element in the third row and first column of the first quantization matrix, which corresponds to the fourth element in the predetermined scanning order; the decoding means is capable of using a quantization matrix in which all elements are set to 16 as the second quantization matrix; the code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used; If the size of the first quantization matrix is 4x4, the size of the second quantization matrix is 4x4. An image decoding device comprising: [Effects of the Invention]

[0009] According to the present invention, it is possible to reduce the amount of code for the quantization matrix more than ever before, thereby improving the coding efficiency of image data. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image encoding device according to a first embodiment. [Figure 2] FIG. 1 is a block diagram showing the configuration of an image decoding device according to a first embodiment. [Figure 3] 4 is a flowchart showing encoding processing in the image encoding device of the first embodiment. [Figure 4]4 is a flowchart showing a decoding process in the image decoding device of the first embodiment. [Figure 5] FIG. 10 is a diagram showing the configuration of computer hardware applied in the second embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the data structure of a bitstream generated by the image encoding device according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of sub-block division in the embodiment. [Figure 8] 5A and 5B are diagrams showing examples of quantization matrices used in the embodiment. [Figure 9] 4A and 4B are diagrams showing a method of scanning elements of a quantization matrix used in the embodiment. [Figure 10] 10A and 10B are diagrams showing difference value matrices of quantization matrices generated in the embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a coding table used for coding difference values of a quantization matrix. [Figure 12] FIG. 2 is a diagram showing an example of a syntax table of a quantization matrix used in the embodiment. [Figure 13] 10A and 10B are diagrams showing examples of reference quantization matrices used in the embodiment. [Figure 14] 10 is a flowchart showing a procedure for encoding a quantization matrix. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0012] [First embodiment] Although the embodiments describe a plurality of features, not all of these features are necessarily essential to the invention, and the plurality of features may be combined in any desired manner. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] 1 is a block diagram of an image encoding device according to the first embodiment. The image encoding device includes a control unit 150 that controls the entire device. The control unit 150 includes a CPU, a ROM that stores programs executed by the CPU, and a RAM that is used as a work area for the CPU. The image encoding device also includes an input terminal 101, a block dividing unit 102, a quantization matrix holding unit 103, a prediction unit 104, a transform / quantization unit 105, an inverse quantization / inverse transform unit 106, an image reproduction unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0014] An input terminal 101 inputs image data to be coded, generated in an image data source, in frame units. The image data source may be of any type, such as an imaging device, a file server or storage medium that stores image data to be coded, etc. An output terminal 112 outputs the coded data to a destination device, which may also be a storage medium, file server, etc.

[0015] The block division unit 102 divides an input frame image into a plurality of basic blocks, and outputs each of the blocks as a basic block to the subsequent prediction unit 104 in order.

[0016] The quantization matrix storage unit 103 generates multiple quantization matrices prior to encoding and stores them in an internal memory (not shown). There are no particular limitations on the method for generating the quantization matrices; for example, the user may input the quantization matrices, the quantization matrices may be calculated based on the characteristics of the input image, or pre-specified initial values may be used. In this embodiment, the quantization matrix storage unit 103 generates and stores two-dimensional quantization matrices 800 to 811 corresponding to orthogonal transformation of 8×8 pixel size, 4×4 pixel size, or 2×2 pixel size shown in FIGS. 8(a) to 8(l). Here, quantization matrices 800, 803, 806, and 809 are for the luminance component, and quantization matrices 801, 802, 804, 805, 807, 808, 810, and 811 are for the two chrominance components.

[0017] The prediction unit 104 determines sub-block division for image data in units of basic blocks, and performs intra-frame prediction (intra-prediction) or inter-frame prediction (inter-prediction) on a sub-block basis to generate predicted image data. Furthermore, the prediction unit 104 calculates and outputs pixel-by-pixel prediction errors from the sub-blocks in the input image data and the corresponding predicted image data. The prediction unit 104 also outputs information necessary for prediction, such as sub-block division, prediction mode, motion vector, etc., along with the prediction errors. Hereinafter, this information necessary for prediction will be referred to as prediction information.

[0018] The transform / quantization unit 105 obtains orthogonal transform coefficients by orthogonally transforming the prediction errors of the sub-blocks input from the prediction unit 104 on a sub-block basis. Furthermore, the transform / quantization unit 105 quantizes the orthogonal transform coefficients using the quantization matrix stored in the quantization matrix storage unit 103, and obtains residual coefficients (quantized orthogonal transform coefficients).

[0019] The inverse quantization and inverse transform unit 106 receives the residual coefficients from the transform and quantization unit 105, performs inverse quantization using the corresponding quantization matrix stored in the quantization matrix storage unit 103, and reproduces orthogonal transform coefficients. The inverse quantization and inverse transform unit 106 further performs inverse orthogonal transform on the orthogonal transform coefficients to reproduce prediction errors.

[0020] The image reproduction unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image reproduction unit 107 generates reproduced image data by adding the prediction error input from the inverse quantization and inverse transform unit 106 to this predicted image data, and stores the reproduced image data in the frame memory 108.

[0021] The in-loop filter 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image stored in the frame memory 108, and stores the filtered image data back in the frame memory 108.

[0022] The encoding unit 110 encodes the residual coefficients output from the transform / quantization unit 105 and the prediction information output from the prediction unit 104 to generate coded data, and outputs the coded data to the integrated coding unit 111 .

[0023] The quantization matrix encoding unit 113 encodes the quantization matrix (see Figures 8(a) to (l)) stored in the quantization matrix storage unit 103 to generate quantization matrix code data, and outputs it to the integrated encoding unit 111.

[0024] The integrated coding unit 111 generates header coded data including the quantization matrix coded data from the quantization matrix coding unit 113. Then, the integrated coding unit 111 appends the coded data output from the coding unit 110 to the header coded data to form a bit stream. Then, the integrated coding unit 111 outputs the formed bit stream via an output terminal 112. The image encoding operation in the image encoding device will now be described in more detail. In this embodiment, moving image data in 4:2:0 color format is input frame by frame at a predetermined frame rate (e.g., 30 frames / second) from input terminal 101, but it may also be configured to input one frame of still image data. For the sake of explanation, this embodiment will be described assuming that block division unit 101 divides the image data input from input terminal 101 into basic blocks of 8×8 pixels. That is, each basic block of 8×8 pixels contains 8×8 pixels of luminance (Y) component and 4×4 pixels of chrominance (Cb and Cr) component pixels. Note that this is for ease of understanding and is not limited to the above numerical values (sizes).

[0025] Before encoding an image, a quantization matrix is generated and encoded.

[0026] The quantization matrix storage unit 103 first generates and stores quantization matrices. Specifically, the quantization matrix is generated according to the size of the sub-block to be coded and the type of prediction method. In this embodiment, the quantization matrix is generated for an 8×8 pixel basic block not divided into sub-blocks as shown in FIG. 7(a), and for a 4×4 pixel sub-block obtained by quadtree-dividing the basic block as shown in FIG. 7(b). That is, the quantization matrix storage unit 103 generates 8×8 pixel and 4×4 pixel quantization matrices for the luminance (Y) component, and 4×4 pixel and 2×2 pixel quantization matrices for the chrominance (Cb and Cr) components. However, the generated quantization matrices are not limited to these, and quantization matrices corresponding to the shape of the sub-block, such as 4×8 or 8×4, may also be generated. The method for determining each element constituting the quantization matrix is not particularly limited. For example, a predetermined initial value may be used, or each element may be individually set. The quantization matrix may also be generated according to the characteristics of the image.

[0027] The quantization matrix storage unit 103 stores the multiple types of quantization matrices generated in this manner in an internal memory (not shown). Fig. 8(a) shows a quantization matrix 800 of 8x8 pixel size for the Y component using intra prediction. Fig. 8(b) also shows a quantization matrix 801 of 4x4 pixel size for the Cb component using intra prediction, and Fig. 8(c) shows a quantization matrix 802 of 4x4 pixel size for the Cr component using intra prediction.

[0028] Similarly, Figure 8(d) shows an 8x8 pixel quantization matrix 803 for the Y component using inter prediction, Figure 8(e) shows a 4x4 pixel quantization matrix 804 for the Cb component using inter prediction, and Figure 8(f) shows a 4x4 pixel quantization matrix 805 for the Cr component using inter prediction.

[0029] 8(g) shows a 4×4 pixel quantization matrix 806 for the Y component using intra prediction, while FIG. 8(h) shows a 2×2 pixel quantization matrix 807 for the Cb component using intra prediction, and FIG. 8(i) shows a 2×2 pixel quantization matrix 808 for the Cr component using intra prediction.

[0030] Similarly, Figure 8(j) shows a 4x4 pixel quantization matrix 809 for the Y component using inter prediction. Also, Figure 8(k) shows a 2x2 pixel quantization matrix 810 for the Cb component using inter prediction, and Figure 8(l) shows a 2x2 pixel quantization matrix 811 for the Cr component using inter prediction.

[0031] For ease of explanation, the quantization matrix is assumed to be composed of 64 pixels (8x8), 16 pixels (4x4), and 4 pixels (2x2), and each square within a bold frame represents an element constituting the quantization matrix. In this embodiment, the 12 types of quantization matrices shown in Figures 8(a) to 8(l) are assumed to be stored in a two-dimensional shape, but the elements in the quantization matrix are not limited to this. Also, depending on the size of the sub-block, it is possible to store multiple quantization matrices for the same color component and the same prediction mode. Generally, quantization matrices realize quantization processing according to human visual characteristics, so as shown in Figures 8(a) to 8(l), the elements in the low-frequency part corresponding to the upper left part of the quantization matrix are small, and the elements in the high-frequency part corresponding to the lower right part are large.

[0032] The quantization matrix encoding unit 113 sequentially reads out the two-dimensional quantization matrices stored in the quantization matrix storage unit 106, and determines the encoding mode of the quantization matrix to be used when encoding each quantization matrix. In this embodiment, three types of quantization matrix encoding modes are used: "quantization matrix reference mode," "inter-quantization matrix difference encoding mode," and "intra-quantization matrix difference encoding mode." For each quantization matrix, the quantization matrix encoding unit 113 determines the quantization matrix encoding mode that minimizes the amount of data in the encoding result.

[0033] Here, the three types of quantization matrix coding modes used in the quantization matrix coding unit 113 of this embodiment will be described in more detail.

[0034] The "quantization matrix reference mode" is a mode used when there is a quantization matrix that matches the quantization matrix to be coded among the coded quantization matrices. When this mode is used, the quantization matrix coding unit 113 codes a quantization matrix reference index, which is an identifier that indicates the coded quantization matrix that matches the coded quantization matrix to be referenced, instead of coding each element of the quantization matrix to be coded.

[0035] Next, the "inter-quantization matrix difference coding mode" is a mode in which, from among the coded quantization matrices, one that is closest to the quantization matrix to be coded is selected, the difference value of each element of each quantization matrix is calculated, and the calculated difference value is coded. When this mode is selected, the quantization matrix coding unit 113 codes a quantization matrix reference index, which is an identifier indicating the quantization matrix to be referenced, and then codes difference information, which is a set of difference values of each element.

[0036] The "intra-quantization matrix difference encoding mode" is used when there is no encoded quantization matrix or when there is no quantization matrix similar to the one to be encoded. The quantization matrix encoding unit scans each element of the quantization matrix to be encoded, calculates the difference value between the elements, and encodes the difference information, which is a set of difference values.

[0037] FIG. 12 is a syntax table used in encoding quantization matrices in this embodiment. Using this syntax table, the encoding process for each quantization matrix shown in FIGS. 8(a) to (l) in this embodiment will be specifically described. First, the structure of the syntax table in FIG. 12 will be described. At the outermost edge of the syntax table, there is a for loop with a parameter called sizeId. sizeId indicates the size of the quantization matrix, and quantization matrices of the same size are encoded consecutively. sizeId=1 indicates a quantization matrix of 2×2 pixels. Similarly, sizeId=2 indicates a quantization matrix of 4×4 pixels, and sizeId=3 indicates a quantization matrix of 8×8 pixels.

[0038] The initial value of sizeId in the for loop is "1," and it is incremented by "1" each time the loop is executed. It is defined that the loop is repeated as long as sizeId is less than 4. In this embodiment, this means that a group of 2x2 pixel quantization matrices is coded, followed by a group of 4x4 pixel quantization matrices, and then a group of 8x8 pixel quantization matrices, and so on. VVC uses orthogonal transforms ranging from the minimum size of 2x2 pixels to 64x64 pixels. It is also possible to configure a for loop ranging from 2x2 pixel size (sizeId=1) to 64x64 pixel size (sizeId=6), and code the corresponding quantization matrices. However, in this embodiment, because the maximum orthogonal transform size, i.e., the size of a sub-block, is 8x8, the upper limit of the for loop is set to sizeId=3, which corresponds to the 8x8 pixel size. By setting the upper limit of the for loop based on the maximum value of the orthogonal transform size actually used in this manner, it is possible to omit coding of unnecessary quantization matrices and prevent the generation of redundant codes.

[0039] Inside the for loop based on sizeId, there is a for loop based on a parameter called matrixId. This matrixId is a parameter based on the prediction mode and color component; a value of 0 indicates intra-prediction for the Y component, a value of 1 indicates intra-prediction for the Cb component, and a value of 2 indicates intra-prediction for the Cr component. Similarly, a value of 3 indicates inter-prediction for the Y component, a value of 4 indicates inter-prediction for the Cb component, and a value of 5 indicates inter-prediction for the Cr component. In other words, among a group of quantization matrices of the same size, the group of quantization matrices corresponding to intra-prediction are coded first, followed by the group of quantization matrices corresponding to inter-prediction. Furthermore, among a group of quantization matrices corresponding to the same prediction mode, the quantization matrices for the Y component, the Cb component, and the Cr component are coded in that order.

[0040] Considering the above coding order of the quantization matrices, the coding order of the quantization matrices in FIGS. 8(a) to (l) is as follows.

[0041] First, 2x2 pixel size images (h), (i), (k), and (l) of Figure 8 are coded. Next, 4x4 pixel size images (g), (b), (c), (j), (e), and (f) of Figure 8 are coded. Finally, 8x8 pixel size images (a) and (d) of Figure 8 are coded.

[0042] The above coding order is essentially (i) the size of the quantization matrix, (ii) the type of intra / inter, and (iii) the type of component (luminance, chrominance). The reason that the size of the quantization matrix is given the first priority is because the coding takes into account the correlation between quantization matrices.

[0043] Next, the encoding procedure for each quantization matrix will be described. First, the quantization matrix encoding unit 113 encodes the determined quantization matrix encoding mode. In the syntax table of FIG. 12, scaling_list_pred_mode_idx corresponds to the quantization matrix encoding mode. In this embodiment, when this value is 0, the "quantization matrix reference mode" is used, when this value is 1, the "inter-quantization matrix difference encoding mode" is used, and when this value is 2, the "intra-quantization matrix difference encoding mode" is used. Since the quantization matrix encoding mode is Golomb encoded, the smaller this value, the smaller the code amount related to scaling_list_pred_mode_idx. That is, in this embodiment, the code amount related to the matrix reference mode is the smallest, which has the advantage of minimizing the generated code amount when there are many identical quantization matrices.

[0044] When the quantization matrix reference mode is used, the quantization matrix encoding unit 113 then encodes a quantization matrix reference index. This corresponds to scaling_list_pred_matrix_id_delta in the syntax table of FIG. 12. For example, the quantization matrix 801 in FIG. 8(b) is identical to the quantization matrix 806 in FIG. 8(g) that is encoded immediately before. Therefore, when encoding the quantization matrix 801 in FIG. 8(b), the quantization matrix encoding unit 113 selects the quantization matrix reference mode and encodes 0 as the quantization matrix reference index, indicating that the quantization matrix is the quantization matrix encoded immediately before. In this way, it is possible to encode a quantization matrix identical to the quantization matrix 806 in FIG. 8(g). Note that if the immediately previous quantization matrix is used, the quantization matrix reference index becomes "1".

[0045] On the other hand, when using the inter-quantization matrix difference coding mode, as in the case of using the quantization matrix reference mode, the quantization matrix coding unit 113 first codes a quantization matrix reference index. Next, the quantization matrix coding unit 113 codes difference information, which is the difference between each element of the reference quantization matrix already coded, determined by the quantization matrix reference index, and each element of the quantization matrix to be coded. This corresponds to scaling_list_pred_delta in the syntax table of FIG. 12. For example, each element of the quantization matrix 802 in FIG. 8(c) is similar to each element of the quantization matrix 801 in FIG. 8(b) that was coded immediately before (in this example, the absolute value of the difference between each element is equal to or less than a predetermined value ("1" or less" in the figure)). Therefore, when coding the quantization matrix 802 in FIG. 8(c), the quantization matrix coding unit 113 selects the inter-quantization matrix difference coding mode and first codes 0, which indicates that the quantization matrix is the quantization matrix coded immediately before, as the quantization matrix reference index. Note that when encoding the difference between each element of the previous quantization matrix, the quantization matrix reference index becomes "1." Next, the quantization matrix encoding unit 113 calculates the difference between each element of the reference quantization matrix in FIG. 8(b) and each element of the quantization matrix to be encoded in FIG. 8(c), thereby calculating the two-dimensional difference value matrix shown in FIG. 10(a). Then, each difference value of the two-dimensional difference value matrix is scanned and arranged in a one-dimensional matrix. FIGS. 9(a) to 9(c) show scanning methods used in this embodiment when arranging a two-dimensional matrix one-dimensionally, with FIG. 9(a) corresponding to an 8×8 quantization matrix, FIG. 9(b) corresponding to a 4×4 quantization matrix, and FIG. 9(c) corresponding to a 2×2 quantization matrix. Since a 4×4 quantization matrix is the target here, the quantization matrix encoding unit 113 uses the scanning method shown in FIG. 9(b) to arrange the two-dimensional difference value matrix of FIG. 10(a) into the one-dimensional difference value matrix shown in FIG. 10(b). Then, the quantization matrix encoding unit 113 encodes each difference value in Fig. 10(b). Note that although the absolute value of each difference value in Fig. 10(b) is equal to or less than "1", there may be cases where the absolute value is greater than 1.By taking such cases into consideration and further obtaining the difference between adjacent difference values, it becomes possible to further reduce the amount of code.

[0046] Furthermore, when using the intra-quantization matrix difference coding mode, the quantization matrix coding unit 113 scans each element of the quantization matrix to be coded using one of FIGS. 9(a) to 9(c), calculates the difference between two consecutive elements, and arranges the calculated difference in a one-dimensional matrix. The quantization matrix coding unit 113 then codes each difference value arranged in the one-dimensional matrix as difference information. This corresponds to scaling_list_delta_coef in the syntax table of FIG. 12. For example, since the quantization matrix 806 in FIG. 8(g) is the first to be coded among the 4×4 pixel quantization matrices, there is no quantization matrix to be referenced using the quantization matrix reference mode or inter-quantization matrix difference coding mode. Therefore, when coding the quantization matrix 806 in FIG. 8(g), the quantization matrix coding unit 113 necessarily selects the intra-quantization matrix difference coding mode. Therefore, the quantization matrix encoding unit 113 scans the quantization matrix 806 of FIG. 8(g) using FIG. 9(b) and calculates the difference between each element and the previous element in the scanning order. The quantization matrix encoding unit 113 arranges the calculated differences in a one-dimensional matrix to obtain the one-dimensional difference value matrix shown in FIG. 10(c). Here, for example, the 4×4 pixel quantization matrix of FIG. 8(g) is scanned using the scanning method shown in FIG. 9(b). After the first element 6 located in the upper left, the element 13 located immediately below it is scanned, and the difference, +7, is calculated. Furthermore, to encode the first element of the quantization matrix (6 in this embodiment), the difference from a predetermined initial value (e.g., 8) is calculated. However, this is not limiting, and the difference from an arbitrary value or the value of the first element itself may also be used. In short, it is sufficient if the initial value is the same as that of the decoding device. The quantization matrix encoding unit 113 then encodes each difference value of FIG. 10(c).

[0047] Here, a supplementary explanation will be given of the encoding process of the above-mentioned quantization matrix. The fifth parameter from the top in Fig. 12, "scaling_list_pred_mode_idx[sizeId][matrixId]", is a parameter indicating the encoding mode of the quantization matrix according to sizeId and matrixId. To calculate this parameter, the quantization matrix specified by sizeId and matrixId (any of Figs. 8(a) to (l)) is encoded according to the above-mentioned three encoding modes, and the encoding mode that generates the smallest amount of code is determined. Then, information specifying the determined encoding mode (0 to 2 in the above example) is set in this parameter.

[0048] The quantization matrix encoding unit 113 generates quantization matrix code data as the encoding result of each quantization matrix generated using each quantization matrix encoding mode as described above. The quantization matrix encoding unit 113 of this embodiment encodes each element of a one-dimensional difference matrix by assigning a code word, which is a binary code, to each element using the encoding table shown in FIG. 11(a), thereby generating quantization matrix code data. Note that the encoding table is not limited to this, and for example, the encoding table shown in FIG. 11(b) may also be used. In this way, the quantization matrix encoding unit 113 outputs the generated quantization matrix code data to the subsequent integrated encoding unit 111.

[0049] Returning to FIG. 1, the integrated encoding unit 111 integrates the coded data of the quantization matrix with the header information required for encoding the image data.

[0050] Next, the image data is coded. Image data for one frame input from an input terminal 101 is supplied to a block division unit 102.

[0051] The block division unit 102 divides the input image data of one frame into a plurality of basic blocks, and outputs the image data in units of basic blocks to the prediction unit 104. In this embodiment, image data in units of basic blocks of 8×8 pixels is supplied to the prediction unit 104.

[0052] The prediction unit 104 performs prediction processing on image data in basic block units input from the block division unit 102. Specifically, it determines sub-block division, which divides the basic block into smaller sub-blocks, and then determines a prediction mode, such as intra prediction or inter prediction, for each sub-block. Intra prediction generates predicted pixels for the block to be coded using coded pixels spatially located around the block to be coded, and also generates an intra prediction mode indicating an intra prediction method, such as horizontal prediction, vertical prediction, or DC prediction. Inter prediction generates predicted pixels for the block to be coded using coded pixels of a frame temporally different from the block to be coded, and also generates motion information indicating reference frames, motion vectors, etc.

[0053] The subblock division method will be explained with reference to Figure 7. The bold frames of blocks 700 to 705 in Figures 7(a) to 7(f) are the same size as the basic blocks, 8x8 pixels. Each square within the bold frame represents a subblock. Figure 7(b) shows an example of conventional square subblock division, in which an 8x8 pixel basic block 701 is divided into four 4x4 pixel subblocks. Figures 7(c) to 7(f) show an example of rectangular subblock division. Figure 7(c) shows that a basic block 702 is divided into two 4x8 pixel subblocks (longitudinal in the vertical direction). Figure 7(d) shows that a basic block 703 is divided into two 8x4 pixel subblocks (longitudinal in the horizontal direction). Figures 7(e) and 7(f) show that basic blocks 704 and 705 are divided into three rectangular subblocks in a 1:2:1 ratio, although the division method is different. In this way, not only square but also rectangular subblocks are used for encoding.

[0054] In this embodiment, we will explain the use of either the method shown in FIG. 7(a), in which an 8×8 pixel basic block is not divided into sub-blocks, or the method shown in FIG. 7(b), in which the sub-blocks are divided into quadtrees. However, the sub-block division method is not limited to this. Ternary tree division as shown in FIG. 7(e) or (f), or binary tree division as shown in FIG. 7(c) or 7(d), may also be used. When sub-block division other than that shown in FIG. 7(a) or 7(b) is used, a quantization matrix corresponding to the sub-block to be used is generated by the quantization matrix storage unit 103. The generated quantization matrix is then coded by the quantization matrix coding unit 113.

[0055] The prediction unit 104 generates predicted image data from the determined prediction mode and the coded area stored in the frame memory 108, calculates a pixel-by-pixel prediction error from the predicted image data corresponding to a sub-block of interest in the input image data, and outputs the error to the transformation and quantization unit 105. The prediction unit 104 also outputs information such as sub-block division and prediction mode as prediction information to the coding unit 110 and the image reproduction unit 107.

[0056] The transform and quantization unit 105 performs orthogonal transform and quantization on the prediction error input from the prediction unit 104 to generate residual coefficients. Specifically, the transform and quantization unit 105 first performs orthogonal transform processing on the prediction error corresponding to the size of the sub-block to generate orthogonal transform coefficients. The transform and quantization unit 105 then quantizes the orthogonal transform coefficients using a quantization matrix stored in the quantization matrix storage unit 103 according to the prediction mode and color component, to generate residual coefficients. In this embodiment, when sub-block division is not performed and intra prediction mode is used, the quantization matrices shown in FIG. 8(a) are used for the orthogonal transform coefficients of the Y component, FIG. 8(b) are used for the orthogonal transform coefficients of the Cb component, and FIG. 8(c) are used for the orthogonal transform coefficients of the Cr component. Similarly, when sub-block division is not performed and inter prediction mode is used instead, the quantization matrices shown in FIG. 8(d) are used for the orthogonal transform coefficients of the Y component, FIG. 8(e) are used for the orthogonal transform coefficients of the Cb component, and FIG. 8(f) are used for the orthogonal transform coefficients of the Cr component. On the other hand, when the sub-block division of FIG. 7(b) is performed and intra prediction mode is used, the quantization matrices of FIG. 8(g) are used for the orthogonal transform coefficients of the Y component, those of FIG. 8(h) are used for the orthogonal transform coefficients of the Cb component, and those of FIG. 8(i) are used for the orthogonal transform coefficients of the Cr component. Similarly, when the sub-block division of FIG. 7(b) is performed but inter prediction mode is used instead, the quantization matrices of FIG. 8(j) are used for the orthogonal transform coefficients of the Y component, those of FIG. 8(k) are used for the orthogonal transform coefficients of the Cb component, and those of FIG. 8(l) are used for the orthogonal transform coefficients of the Cr component. However, the quantization matrices used are not limited to these. The generated residual coefficients and chrominance integration information are output to the encoding unit 110 and the inverse quantization and inverse transform unit 106.

[0057] The inverse quantization and inverse transform unit 106 regenerates orthogonal transform coefficients by inverse quantizing the residual coefficients input from the transform and quantization unit 105 using the corresponding quantization matrix stored in the quantization matrix storage unit 103. The inverse quantization and inverse transform unit 106 then performs inverse orthogonal transform on the regenerated orthogonal transform coefficients to regenerate prediction errors. As with the transform and quantization unit 105, the inverse quantization process uses a quantization matrix corresponding to the size and color components of the sub-block to be coded. Specifically, the inverse quantization and inverse transform unit 106 performs inverse quantization using the same quantization matrix as used by the transform and quantization unit 105. That is, when sub-block division is not performed and intra prediction mode is used, the quantization matrix shown in FIG. 8(a) is used for the orthogonal transform coefficients of the Y component, the quantization matrix shown in FIG. 8(b) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix shown in FIG. 8(c) is used for the orthogonal transform coefficients of the Cr component. Similarly, when subblock division is not performed and inter prediction mode is used instead, the quantization matrix of Figure 8(d) is used for the orthogonal transform coefficients of the Y component, the quantization matrix of Figure 8(e) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix of Figure 8(f) is used for the orthogonal transform coefficients of the Cr component. On the other hand, when the subblock division of Figure 7(b) is performed and intra prediction mode is used, the quantization matrix of Figure 8(g) is used for the orthogonal transform coefficients of the Y component, the quantization matrix of Figure 8(h) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix of Figure 8(i) is used for the orthogonal transform coefficients of the Cr component. Similarly, when the subblock division of Figure 7(b) is performed and inter prediction mode is used instead, the quantization matrix of Figure 8(j) is used for the orthogonal transform coefficients of the Y component, the quantization matrix of Figure 8(k) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix of Figure 8(l) is used for the orthogonal transform coefficients of the Cr component.

[0058] The orthogonal transform coefficients thus reconstructed are subjected to inverse orthogonal transform to reconstruct prediction errors, which are then output to the image reconstruction unit 107 .

[0059] The image reproduction unit 107 appropriately refers to the frame memory 108 and reproduces a predicted image based on the prediction information input from the prediction unit 104. The image reproduction unit 107 then generates reproduced image data for each sub-block based on the reproduced predicted image and the prediction error of the sub-block reproduced by the inverse quantization and inverse transform unit 106, and stores the reproduced image data in the frame memory 108.

[0060] The in-loop filter unit 109 reads the reproduced image data from the frame memory 108 and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 109 stores the filtered image data back in the frame memory 108.

[0061] The encoding unit 110 entropy-encodes the residual coefficients and chrominance integrated information for each subblock generated by the transform / quantization unit 105, as well as the prediction information input from the prediction unit 104, to generate coded data. There is no particular specification as to the entropy coding method, but Golomb coding, arithmetic coding, Huffman coding, etc. can be used. The encoding unit 110 outputs the generated coded data to the integrated coding unit 111.

[0062] The integrated coding unit 111 multiplexes the coded data of the header and the coded data input from the coding unit 110 to form a bit stream. Then, the integrated coding unit 111 outputs the formed bit stream from an output terminal 112 to an external device (such as a storage medium or a network).

[0063] FIG. 6(a) shows an example of the data structure of a bitstream output in this embodiment. The sequence header includes coded data of quantization matrices and is configured with the coded results of each quantization matrix. However, the location where the coded data of the quantization matrices is stored is not limited to this, and it may be arranged in a picture header section as shown in FIG. 6(b) or a header section spanning multiple pictures. Furthermore, when changing a quantization matrix within a sequence, it is also possible to update it by newly coding the quantization matrix. In this case, all quantization matrices may be rewritten, or it is also possible to change only a portion of the quantization matrix by specifying the size, prediction mode, and color components of the quantization matrix corresponding to the quantization matrix to be rewritten.

[0064] FIG. 3 is a flowchart showing the encoding process for one frame performed by the control unit 150 in the image encoding device according to the embodiment.

[0065] First, prior to encoding an image, in S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold two-dimensional quantization matrices. The quantization matrix holding unit 103 of this embodiment corresponds to blocks of 8×8 to 2×2 pixel sizes, and generates and holds quantization matrices corresponding to the respective color components and prediction modes shown in Figures 8(a) to 8(l).

[0066] In S302, the control unit 150 controls the quantization matrix encoding unit 113 to encode the quantization matrix generated and held in S301. The specific operation of the quantization matrix encoding unit 113 has already been described, and will not be described again. In this embodiment, the control unit 150 controls the quantization matrix encoding unit 113 to encode the quantization matrices 801 to 812 shown in FIGS. 8(a) to 8(l) based on the syntax table in FIG. 12, thereby generating quantization matrix code data.

[0067] In S303, the control unit 150 controls the integration encoding unit 111 to encode and output the generated quantization matrix code data as well as header information required for encoding the image data.

[0068] In S304, control unit 150 controls block dividing unit 102 to divide the input image in units of frames into units of basic blocks.

[0069] In S305, the control unit 150 controls the prediction unit 104 to divide the image data of each basic block generated in S304 into sub-blocks and to perform prediction processing for each sub-block, thereby generating prediction information such as sub-block division information and a prediction mode, as well as predicted image data. Furthermore, the control unit 150 controls the prediction unit 104 to calculate a prediction error from the input sub-block image data and predicted image data.

[0070] In S306, the control unit 150 controls the transform / quantization unit 105 to perform orthogonal transform on the prediction errors calculated in S305 and generate orthogonal transform coefficients. Furthermore, the control unit 150 controls the transform / quantization unit 105 to perform quantization using the quantization matrix generated and stored in S301 and generate residual coefficients. In this embodiment, the quantization matrices shown in Figures 8(a) to 8(l) are used depending on the size, prediction mode, and color components of the sub-blocks.

[0071] In S307, the control unit 150 controls the inverse quantization and inverse transform unit 106 to inverse quantize the residual coefficients generated in S306 using the quantization matrix generated and stored in S301, thereby reproducing orthogonal transform coefficients. In this step, the same quantization matrix as used in S306 is used for the inverse quantization process. Then, the reproduced orthogonal transform coefficients are subjected to inverse orthogonal transform to reproduce the prediction error.

[0072] In S308, the control unit 150 controls the image reproduction unit 107 to reproduce a predicted image based on the prediction information generated in S305, reproduce image data from the reproduced predicted image and the prediction error generated in S307, and store the image data in the frame memory 108.

[0073] In S309, the control unit 150 controls the encoding unit 110 to encode the prediction information generated in S305 and the residual coefficients generated in S306, thereby generating coded data. The encoding unit 110 outputs the generated coded data to the integrative encoding unit 111. The integrative encoding unit 111 positions the coded data from the encoding unit 110 so that it follows the header generated earlier, and outputs the coded data.

[0074] In S310, the control unit 150 determines whether or not the encoding of all basic blocks in the frame of interest has been completed. If the control unit 150 determines that the encoding has been completed, the process proceeds to S311. If the control unit 150 determines that there are still basic blocks to be encoded, the process returns to S304, and the encoding of the next basic block continues.

[0075] In S311, the control unit 150 controls the in-loop filter unit 109 to perform in-loop filtering on the image data reproduced in S308, generate a filtered image, and then terminate the processing.

[0076] With the above configuration and operation, particularly in S302, the amount of code generated by the quantization matrix can be reduced by determining the quantization matrix coding mode that minimizes the amount of code generated by the quantization matrix and performing coding.

[0077] In this embodiment, in the quantization matrix reference mode and the inter-quantization matrix difference encoding mode, only quantization matrices of the same size can be referenced, but a configuration in which quantization matrices of different sizes can be referenced using scaling may also be used. For example, when encoding quantization matrix 806 in FIG. 8(g), it may be enlarged by referring to quantization matrix 807 in FIG. 8(h), and the difference value between the two may be encoded as difference information. This makes it possible to further reduce the amount of data of the quantization matrix that is first encoded for each size.

[0078] In addition, in this embodiment, as shown in the syntax table of Fig. 12, the quantization matrices are coded in ascending order from smallest to largest, but it is also possible to code the quantization matrices in descending order from largest to largest. This makes it possible to further reduce the amount of data in the quantization matrices, especially when reference between quantization matrices of different sizes is possible as described above.

[0079] Furthermore, in this embodiment, the reference target of the quantization matrix is limited to an encoded quantization matrix, but it is also possible to use a flat quantization matrix as a reference when no quantization matrix is used, as shown in Figures 13(a) to 13(c). This makes it possible to further reduce the amount of data in the quantization matrix, especially when the quantization matrix to be encoded is close to a flat quantization matrix.

[0080] 2 is a block diagram of an image decoding device that decodes the coded image data generated by the image coding device. The configuration and operation related to the decoding process will be described below with reference to this figure.

[0081] The image decoding device has a control unit 250 that controls the entire device. The control unit 250 has a CPU, a ROM that stores programs executed by the CPU, and a RAM that is used as a work area for the CPU. The image decoding device also has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization / inverse transform unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0082] An input terminal 201 is used to input an encoded bit stream, and the input source may be, for example, a storage medium that stores an encoded stream, but may also be input from a network, and the type of input source is not important.

[0083] The separate decoding unit 202 separates the bitstream into coded data related to information on decoding processing and coefficients, and decodes the coded data present in the header part of the bitstream. The separate decoding unit 202 of this embodiment separates quantization matrix coded data and outputs it to a quantization matrix decoding unit 209. The separate decoding unit 202 also outputs image coded data to a decoding unit 203. In other words, the separate decoding unit 202 performs the reverse operation of the integrated coding unit 111 in FIG. 1.

[0084] The quantization matrix decoding unit 209 reproduces and stores the quantization matrix by decoding the quantization matrix code data supplied from the separating decoding unit 202.

[0085] The decoding unit 203 decodes the coded data of the image output from the separate decoding unit 202, and reproduces residual coefficients and prediction information in units of sub-blocks.

[0086] Similar to the inverse quantization and inverse transform unit 106 in FIG. 1 , the inverse quantization and inverse transform unit 204 uses the reconstructed quantization matrix to perform inverse quantization on the residual coefficients of the target sub-block to obtain the inversely quantized coefficients, and then performs inverse orthogonal transform to reconstruct the prediction error.

[0087] The image reproduction unit 205 generates predicted image data based on the input prediction information by appropriately referring to the frame memory 206. The image reproduction unit 205 then generates reproduced image data of the target sub-block from this predicted image data and the prediction error reproduced by the inverse quantization and inverse transform unit 204, and stores the reproduced image data in the frame memory 206.

[0088] Similar to the in-loop filter unit 109 in FIG. 1, the in-loop filter unit 207 performs in-loop filtering such as deblocking filtering on the reproduced image data stored in the frame memory 206, and stores the image data after filtering back into the frame memory 206.

[0089] The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. The output destination is generally a display device, but may be other devices.

[0090] The operation of the image decoding device of the above embodiment relating to image decoding will now be described in more detail. In this embodiment, the coded bit stream is input in units of frames.

[0091] 2, a bitstream for one frame input from an input terminal 201 is supplied to a separate decoding unit 202. The separate decoding unit 202 separates coded data related to information about decoding processing and coefficients from the bitstream and decodes the coded data present in the header portion of the bitstream. The separate decoding unit 202 then supplies the quantization matrix coded data contained in the header portion to a quantization matrix decoding unit 209, and supplies coded data of image data to a decoding unit 203. Specifically, the separate decoding unit 202 first extracts quantization matrix coded data from the sequence header of the bitstream shown in FIG. 6(a), and outputs the extracted quantization matrix coded data to the quantization matrix decoding unit 209. In this embodiment, quantization matrix coded data corresponding to the quantization matrices shown in FIGS. 8(a) to 8(l) is extracted and output. Subsequently, coded data for each basic block of picture data is extracted and output to the decoding unit 203.

[0092] The quantization matrix decoding unit 209 first decodes the input quantization matrix code data and reproduces the coding results of each quantization matrix generated on the coding side based on the syntax table shown in Fig. 12. As shown in Fig. 6(a), the coding results of each quantization matrix are made up of a quantization matrix coding mode and data corresponding to each quantization matrix coding mode.

[0093] For example, if the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table of FIG. 12, is 0, this means that the quantization matrix has been coded in quantization matrix reference mode. In this case, the coding result of the quantization matrix consists of a quantization matrix coding mode (0) and a quantization matrix reference index. For example, when decoding a 4×4 quantization matrix for intra prediction Cb components, the quantization matrix decoding unit 209 first reproduces quantization matrix coding mode = 0, which indicates the quantization matrix reference mode. Then, the quantization matrix decoding unit 209 reproduces the quantization matrix reference index. If the quantization matrix reference index indicates that the quantization matrix to be referenced among the decoded quantization matrices is the one decoded immediately before, then the quantization matrix reference index = 0. If the quantization matrix reference index = 1, the quantization matrix decoded one step earlier is referenced. From this information, the quantization matrix decoding unit 209 determines that the 4x4 quantization matrix for intra prediction Cb component is the same as the quantization matrix 806 for intra prediction Y component in Figure 8(g), and reproduces the quantization matrix 801 in Figure 8(b).

[0094] Furthermore, when the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table of FIG. 12, is 1, this means that the quantization matrix has been coded in inter-quantization matrix differential coding mode. In this case, the coding result of the quantization matrix is composed of the quantization matrix coding mode (1), a quantization matrix reference index, and differential information. For example, when decoding a 4×4 quantization matrix for intra prediction Cr components, the quantization matrix decoding unit 209 first reproduces quantization matrix coding mode = 1, which indicates inter-quantization matrix differential coding mode. Then, the quantization matrix decoding unit 209 reproduces the quantization matrix reference index. If the quantization matrix reference index indicates that the quantization matrix to be referenced among the decoded quantization matrices is the one decoded immediately before, the quantization matrix reference index = 0. If the quantization matrix reference index = 1, the quantization matrix decoded one step earlier is referenced. Next, the quantization matrix decoding unit 209 decodes the difference information, i.e., scaling_list_pred_delta in the syntax table of Fig. 12, for the number of elements of the quantization matrix, to regenerate the one-dimensional difference matrix shown in Fig. 10(b). The quantization matrix decoding unit 209 arranges the one-dimensional difference matrix of Fig. 10(b) two-dimensionally using the scanning method of Fig. 9(b), to obtain the two-dimensional difference matrix shown in Fig. 10(a). The quantization matrix decoding unit 209 then adds each difference value of the two-dimensional difference matrix of Fig. 10(a) to each element of the quantization matrix 801 of Fig. 8(b), which is the reference target, to regenerate the quantization matrix 802 of Fig. 8(c).

[0095] On the other hand, if the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table of FIG. 12, is 2, this means that the quantization matrix has been coded in the intra-quantization matrix differential coding mode. In this case, the coding result of the quantization matrix is composed of the quantization matrix coding mode (2) and differential information. For example, when decoding a 4x4 quantization matrix for intra-prediction Y component, the quantization matrix decoding unit 209 first reproduces the quantization matrix coding mode = 2, which indicates the intra-quantization matrix differential coding mode. Then, the quantization matrix decoding unit 209 decodes the differential information, i.e., scaling_list_delta_coef in the syntax table of FIG. 12, for the number of elements of the quantization matrix, to reproduce the one-dimensional differential matrix shown in FIG. 10(c). The quantization matrix decoding unit 209 adds each difference value in the one-dimensional difference matrix of Figure 10(b) to the previous element, arranges them two-dimensionally using the scanning method of Figure 9(b), and regenerates the quantization matrix 806 of Figure 8(g).

[0096] In this embodiment, the coding table of FIG. 11(a) is used to decode the difference information, but the coding table of FIG. 11(b) may also be used. In short, it is sufficient to use the same one as on the coding side. The quantization matrix decoding unit 209 then stores the quantization matrices 800 to 811 reproduced in this way. Here, the operation performed is the reverse of the operation of the quantization matrix coding unit 113 on the coding side.

[0097] The decoding unit 203 decodes the coded data supplied from the separate decoding unit 202, reproduces prediction information, and further reproduces residual coefficients. First, the decoding unit 203 reproduces the prediction information and obtains the prediction mode used in the sub-block. The decoding unit 203 outputs the reproduced residual coefficients to the inverse quantization and inverse transform unit 204, and outputs the reproduced prediction information to the image reproduction unit 205.

[0098] The inverse quantization and inverse transform unit 204 performs inverse quantization on the input residual coefficients using the quantization matrix reproduced by the quantization matrix decoding unit 209 to generate orthogonal transform coefficients, and then performs inverse orthogonal transform to reproduce prediction errors. Like the inverse quantization and inverse transform unit 106 on the encoding side, the inverse quantization and inverse transform unit 204 performs inverse quantization using a quantization matrix corresponding to the size and color components of the sub-block to be decoded. That is, when sub-block division is not performed and intra prediction mode is used, the quantization matrices shown in Figure 8(a) are used for the orthogonal transform coefficients of the Y component, Figure 8(b) are used for the orthogonal transform coefficients of the Cb component, and Figure 8(c) are used for the orthogonal transform coefficients of the Cr component. Similarly, when sub-block division is not performed and inter prediction mode is used instead, the quantization matrices shown in Figure 8(d) are used for the orthogonal transform coefficients of the Y component, Figure 8(e) are used for the orthogonal transform coefficients of the Cb component, and Figure 8(f) are used for the orthogonal transform coefficients of the Cr component. On the other hand, when the subblock division of Figure 7(b) is performed and intra prediction mode is used, the quantization matrix of Figure 8(g) is used for the orthogonal transform coefficients of the Y component, the quantization matrix of Figure 8(h) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix of Figure 8(i) is used for the orthogonal transform coefficients of the Cr component.Furthermore, when the subblock division of Figure 7(b) is similarly performed and inter prediction mode is used instead, the quantization matrix of Figure 8(j) is used for the orthogonal transform coefficients of the Y component, the quantization matrix of Figure 8(k) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix of Figure 8(l) is used for the orthogonal transform coefficients of the Cr component.

[0099] The orthogonal transform coefficients thus reconstructed are subjected to inverse orthogonal transform, and the reconstructed prediction errors are output to the image reconstruction unit 205. However, the quantization matrix used is not limited to this, and it may be the same as the quantization matrix used in the transform / quantization unit 105 and the inverse quantization / inverse transform unit 106 on the encoding side.

[0100] The image reproduction unit 205 appropriately refers to the frame memory 206 based on the prediction information input from the decoding unit 203, and reproduces a predicted image. The image reproduction unit 205 of this embodiment uses intra prediction or inter prediction, similar to the prediction unit 104 on the encoding side. The specific prediction process is the same as that of the prediction unit 104 on the encoding side, and therefore description thereof will be omitted. The image reproduction unit 205 reproduces image data from this predicted image and the prediction error input from the inverse quantization and inverse transform unit 204, and stores the image data in the frame memory 206. The stored image data is used as a reference during prediction.

[0101] Similar to the in-loop filter unit 109 on the encoding side, the in-loop filter unit 207 reads out a reconstructed image from the frame memory 206 and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 207 stores the filtered image back in the frame memory 206.

[0102] The reproduced image stored in the frame memory 206 is ultimately output from an output terminal 208 to an external device (typically a display device).

[0103] FIG. 4 is a flowchart showing the decoding process of the control unit 250 in the image decoding device according to the embodiment.

[0104] First, in S401, the control unit 250 controls the separate decoding unit 202 to separate information related to the decoding process and coded data related to coefficients from the bitstream, and decodes the coded data of the header part. More specifically, the separate decoding unit 202 supplies the coded data of the quantization matrix to the quantization matrix decoding unit 209, and supplies the coded data of the image to the decoding unit 203.

[0105] In S402, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix code data reproduced in S401 based on the syntax table of Fig. 12, and reproduce and store the quantization matrices 801 to 811 shown in Fig. 8(a) to (l). The specific operation of the quantization matrix decoding unit 209 has already been explained, so a description thereof will be omitted.

[0106] In S403, the control unit 250 controls the decoding unit 203 to decode the coded data separated in S401, reproduce the prediction information, and reproduce the residual coefficients.

[0107] In S404, the control unit 250 controls the inverse quantization and inverse transform unit 204 to perform inverse quantization on the residual coefficients using the quantization matrix reproduced in S402, thereby generating orthogonal transform coefficients. The inverse quantization and inverse transform unit 204 further performs inverse orthogonal transform to reproduce prediction errors. In this embodiment, the quantization matrix to be used in the inverse quantization process is determined according to the color components and size of the sub-block to be decoded. That is, the inverse quantization and inverse transform unit 204 performs inverse quantization using the quantization matrices shown in Figures 8(a) to 8(l) according to the size, prediction mode, and color components of the sub-block. However, the quantization matrix to be used is not limited to these, and may be the same as the quantization matrix used on the encoding side.

[0108] In S405, the control unit 250 controls the image reproduction unit 205 to reproduce an image based on the prediction information generated in S403. Specifically, the image reproduction unit 205 reproduces a predicted image by referring to the frame memory 206 based on the prediction information. At this time, the image reproduction unit 205 uses intra prediction or inter prediction, as in S305 on the encoding side. Then, the image reproduction unit 205 reproduces image data from the reproduced predicted image and the prediction error generated in S404, and stores the reproduced image data in the frame memory 206.

[0109] In S406, the control unit 250 determines whether decoding of all basic blocks in the frame of interest has been completed. If so, the process proceeds to S407; if there are any uncoded basic blocks, the process returns to S403 to decode the next basic block.

[0110] In S407, the control unit 250 controls the in-loop filter unit 207 to perform in-loop filtering on the image data reproduced in S405, generate a filtered image, and then terminate the processing.

[0111] With the above configuration and operation, it is possible to decode the coded bitstream generated by the image coding device described above, i.e., a bitstream coded in an appropriate quantization matrix coding mode in which the amount of code generated by the quantization matrix is suppressed.

[0112] In this embodiment, in the quantization matrix reference mode and inter-quantization matrix difference encoding mode, only quantization matrices of the same size can be referenced, but a configuration in which quantization matrices of different sizes can be referenced using scaling may also be used. For example, when decoding quantization matrix 806 in FIG. 8(g), it may be enlarged by referring to quantization matrix 807 in FIG. 8(h), and the difference value between the two may be decoded as difference information to reproduce the quantization matrix. This makes it possible to decode a bitstream in which the amount of data of the quantization matrix initially encoded at each size is further reduced.

[0113] In addition, in this embodiment, as shown in the syntax table of Fig. 12, the quantization matrices are decoded in order from smallest to largest, but it may also be decoded in order from largest to largest. This makes it possible to decode a bitstream with a further reduction in the amount of quantization matrix data, particularly when reference between quantization matrices of different sizes is possible as described above.

[0114] Furthermore, in this embodiment, the reference target of the quantization matrix is limited to the decoded quantization matrix, but it is also possible to use a flat quantization matrix as a reference when no quantization matrix is used, as shown in Figures 13(a) to 13(c). This makes it possible to decode a bitstream with a further reduction in the amount of quantization matrix data, particularly when the quantization matrix to be decoded is close to a flat quantization matrix.

[0115] [Second embodiment] Although the processing units of the image encoding device and the image decoding device in the first embodiment have been described as being configured as hardware, the processing performed by each processing unit shown in these figures may be configured as a computer program.

[0116] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer applicable to the image encoding device and decoding device according to the above embodiment.

[0117] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and ROM 503, and also executes the processes described above as being performed by the image processing device according to the embodiment. That is, the CPU 501 functions as each of the processing units shown in FIGS. 1 and 2.

[0118] The RAM 502 has an area for temporarily storing data acquired from the outside via an external storage device 506 and an I / F (interface) 507. The RAM 502 is also used as a work area used when the CPU 501 executes various processes. The RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as needed.

[0119] The ROM 503 stores setting data for the computer, a boot program, etc. The operation unit 504 is made up of a keyboard, a mouse, etc., and can be operated by a user of the computer to input various instructions to the CPU 501. The display unit 505 displays the results of processing by the CPU 501. The display unit 505 is made up of, for example, a liquid crystal display.

[0120] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores an OS (operating system) and computer programs (application programs) that cause the CPU 501 to implement the functions of the various units shown in Figures 1 and 2. Furthermore, the external storage device 506 may also store image data to be processed.

[0121] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501 and become the subject of processing by the CPU 501. An I / F 507 can be connected to a network such as a LAN or the Internet, or other devices such as a projector or display device, and the computer can obtain and send various information via this I / F 507. 508 is a bus that connects the above-mentioned components.

[0122] In the above configuration, when the device is powered on, the CPU 501 executes a boot program stored in the ROM 503, and loads and executes an OS stored in the external storage device 506 into the RAM 502. Then, under the control of the OS, the CPU 501 loads an application program related to encoding or decoding from the external storage device 506 into the RAM 502 and executes it. As a result, the CPU 501 functions as each processing unit in FIG. 1 or FIG. 2, and the device functions as an image encoding device or an image decoding device.

[0123] Here, the encoding process of the quantization matrix in the second embodiment, that is, the process corresponding to S302 in FIG. 3, will be described with reference to the flowchart in FIG.

[0124] The order of the quantization matrices to be coded is the same as in the first embodiment, starting with 2x2 pixel size quantization matrices 807 → 808 → 810 → 811, then 4x4 pixel size quantization matrices 806 → 801 → 802 → 809 → 804 → 806, and finally 8x8 pixel size quantization matrices 800 → 803.

[0125] Moreover, the variables sizeId and matrixId shown below have the same meanings as those shown in the first embodiment.

[0126] That is, the variable sizeId indicates the size of the quantization matrix, and when sizeId is "1", it indicates that the size of the quantization matrix to be coded is 2x2 pixels. When sizeId is "2", it indicates that the size of the quantization matrix to be coded is 4x4 pixels. When sizeId is "3", it indicates that the size of the quantization matrix to be coded is 8x8 pixels. Furthermore, the variable matrixId indicates the order of quantization matrices of the same size. The quantization matrix to be coded is identified by sizeId and matrixId.

[0127] For example, when sizeId=2 and matrixId=0, the first quantization matrix 806 of 4×4 pixel size is the quantization matrix to be coded.

[0128] First, in S1401, the CPU 501 sets the variable sizeId to an initial value of "1," thereby setting the size of the quantization matrix to be coded to 2x2 pixels. Next, in S1402, the CPU 501 sets matrixId to an initial value of "0." As a result, when S1402 is executed for the first time, the quantization matrix 807 in Fig. 8 is set as the quantization matrix of interest to be coded.

[0129] In S1403, the CPU 501 determines whether matrixId is "0". If matrixId is other than "0", it means that an encoded quantization matrix of the same size as the quantization matrix of interest exists. Therefore, the CPU 501 proceeds to S1404 and performs search processing in the quantization matrix reference mode. In other words, the CPU 501 performs search processing to determine whether an encoded quantization matrix identical to the quantization matrix of interest exists.

[0130] In S1405, the CPU 501 determines whether or not an encoded quantization matrix identical to the quantization matrix of interest was found. If it is determined that the search was successful, the CPU 501 proceeds to S1406, where it outputs "0" and a value specifying the matching quantization matrix as the encoded data of the quantization matrix of interest. Setting "0" to the beginning of the encoded data is equivalent to setting the scaling_list_pred_mode_idx parameter to "0." In addition, in this embodiment, since the number of quantization matrices of 4x4 pixel size, "6," is greater than the number of quantization matrices of other sizes, the "value specifying the matching quantization matrix" requires only three bits to be assigned to the value specifying the quantization matrix. On the other hand, if it is determined that the search was not successful, the CPU 501 proceeds to S1407.

[0131] If the CPU 501 determines in step S1403 that matrixId is "0", the process proceeds to step S1409.

[0132] In S1407, the CPU 501 executes the inter-quantization matrix difference coding mode for the quantization matrix of interest. Then, in S1408, the CPU 501 obtains "1" and coded data in the inter-quantization matrix difference coding mode. Setting "1" at the beginning of the coded data is equivalent to setting the scaling_list_pred_mode_idx parameter to "1." Information indicating the order of the quantization matrix that is the same as the quantization matrix of interest may be output following the initial "1," and then the coded differential data may be output.

[0133] Next, at S1409, the CPU 501 executes the intra-quantization matrix difference coding mode for the quantization matrix of interest. Then, at S1410, the CPU 501 obtains "2" and coded data in the inter-quantization matrix difference coding mode. Setting "2" to the beginning of the coded data is equivalent to setting the return value of scaling_list_pred_mode_idx to "2."

[0134] Then, in S1411, the CPU 501 outputs the smaller amount of coded data generated in S1408 or S1410 as coded data for the quantization matrix of interest. Note that if the process branches from S1403 to S1409, coded data generated in the intra-quantization matrix difference coding mode in S1409 is output.

[0135] At S1412, the CPU 501 increments the variable matrixId by "1." Then, at S1413, the CPU 501 determines, based on the variables sizeId and matrixId, whether or not encoding of all quantization matrices of the same size as the quantization matrix of interest has been completed. If the determination result at S1413 is "No," the CPU 501 proceeds to S1403 and repeats the above process.

[0136] If the determination result in S1413 is "Yes", the CPU 501 increments sizeId by "1". Then, in S1415, the CPU 501 determines whether or not encoding of quantization matrices of all sizes has been completed, based on the variables sizeId and matrixId. If the determination result in S1415 is "No", this means that quantization matrices of unencoded sizes exist, and the CPU returns the process to S1402. If the determination result in S1415 is "Yes", the CPU ends this process (S302 in FIG. 3).

[0137] It should be noted that when the process proceeds to S1406, it is guaranteed that the amount of coded data in the quantization matrix reference mode will be reduced, so comparison with coded data in other modes has been omitted, but of course comparison may also be made.

[0138] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Industrial Applicability]

[0139] The present invention is applicable to encoding and decoding devices that encode and decode still and moving images, and is particularly applicable to encoding and decoding methods that use quantization matrices. [Explanation of symbols]

[0140] 101...input terminal, 102...block division unit, 103...quantization matrix storage unit, 104...prediction unit, 105...transformation and quantization unit, 106...inverse quantization and inverse transformation unit, 107...image reproduction unit, 108...frame memory, 109...in-loop filter unit, 110...encoding unit, 111...integrated encoding unit, 112...output terminal, 113...quantization matrix encoding unit, 150...control unit

Claims

1. decoding means for decoding the first quantization matrix; and an inverse quantization means for inversely quantizing residual coefficients using the first quantization matrix, the decoding means decodes first information representing a mode of decoding of the first quantization matrix from a scaling_list_data syntax structure; When the decoded first information indicates that a first mode is used, the decoding means decodes second information for specifying a second quantization matrix to be referenced from the scaling_list_data syntax structure, and derives the first quantization matrix by determining that the second quantization matrix specified based on the second information is identical to the first quantization matrix; If the decoded first information indicates that a second mode is used, the decoding means decodes the second information to identify the second quantization matrix from the scaling_list_data syntax structure; (a) associating a first element, which is a first element in a first one-dimensional matrix consisting of a plurality of elements, with the first element being a difference value element between the element in the first row and the first column of the first quantization matrix and the element in the first row and the first column of the second quantization matrix specified based on the second information, with the first row and the first column of a two-dimensional matrix as a first element in a predetermined scanning order, and deriving the element in the first row and the first column of the first quantization matrix using the first element associated with the first row and the first column of the second quantization matrix; (b) associating a second element in the first one-dimensional matrix, the second element being a difference value element between the element in the second row, the first column of the first quantization matrix and the element in the second row, the first column of the second quantization matrix, with the second row, the first column of the two-dimensional matrix as a second element in the predetermined scanning order, and deriving the element in the second row, the first column of the first quantization matrix using the second element associated with the second row, the first column and the element in the second row, the first column of the second quantization matrix; (c) associating a third element in the first one-dimensional matrix, the third element being a difference value element between the element in the first row, second column of the first quantization matrix and the element in the first row, second column of the second quantization matrix, with the first row, second column of a two-dimensional matrix as a third element in the predetermined scanning order, and decoding the first quantization matrix by using the third element associated with the first row, second column and the element in the first row, second column of the second quantization matrix to derive the element in the first row, second column of the first quantization matrix; When the decoded first information indicates that a third mode is used, the decoding means decodes, from the scaling_list_data syntax structure, a plurality of difference values including a difference value between an element in the first row and first column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order; (a) using a first element in a second one-dimensional matrix obtained from the plurality of difference values, deriving an element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order; (b) using a second element in the second one-dimensional matrix, deriving an element in the second row and first column of the first quantization matrix corresponding to the second element in the predetermined scanning order; (c) using a third element in the second one-dimensional matrix, deriving an element in the first row and second column of the first quantization matrix that corresponds to the third element in the predetermined scanning order; (d) decoding the first quantization matrix by using a fourth element in the second one-dimensional matrix to derive an element in the third row and first column of the first quantization matrix, which corresponds to the fourth element in the predetermined scanning order; the decoding means is capable of using a quantization matrix in which all elements are set to 16 as the second quantization matrix; a code amount of the first information indicating that the first mode is used is smaller than a code amount of the first information indicating that the second mode is used; If the size of the first quantization matrix is 4x4, the size of the second quantization matrix is 4x4. An image decoding device comprising:

2. quantization means for quantizing transform coefficients of blocks in an image using a first quantization matrix; encoding means for encoding the first quantization matrix; When a first mode is used to encode the first quantization matrix, the encoding means encodes information indicating that the first quantization matrix is the same as a reference second quantization matrix and information for identifying the second quantization matrix into a scaling_list_data syntax structure; When a second mode is used to encode the first quantization matrix, the encoding means encodes information for identifying the second quantization matrix in the scaling_list_data syntax structure; (a) a first element, which is the first element in a predetermined scanning order, and which is an element of a difference value between the element in the first row and the first column of the first quantization matrix and the element in the first row and the first column of the second quantization matrix; (b) a second element in the predetermined scanning order, the second element being a difference value element between the element in the second row and the first column of the first quantization matrix and the element in the second row and the first column of the second quantization matrix; (c) encoding information used to derive a first one-dimensional matrix consisting of a plurality of elements including a third element in the predetermined scanning order, the third element being an element of a difference value between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix; When a third mode is used to encode the first quantization matrix, the encoding means (a) encoding a difference value between the element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order and a predetermined value into the scaling_list_data syntax structure; (b) encoding a difference value between the element in the second row and the first column of the first quantization matrix corresponding to the second element in the predetermined scanning order and the element in the first row and the first column of the first quantization matrix into the scaling_list_data syntax structure; (c) encoding a difference value between the element in the first row, second column of the first quantization matrix corresponding to the third element in the predetermined scanning order and the element in the second row, first column of the first quantization matrix into the scaling_list_data syntax structure; (d) encoding a difference value between an element in the third row and first column of the first quantization matrix corresponding to the fourth element in the predetermined scanning order and an element in the first row and second column of the first quantization matrix into the scaling_list_data syntax structure; encoding a difference value between two consecutive elements in the predetermined scanning order of the first quantization matrix; the encoding means is capable of using a quantization matrix in which all elements are set to 16 as the second quantization matrix; The code amount of the information indicating that the first mode is used is smaller than the code amount of the information indicating that the second mode is used. If the size of the first quantization matrix is 4x4, the size of the second quantization matrix is 4x4. An image encoding device comprising:

3. a decoding step of decoding the first quantization matrix; a dequantization step of dequantizing residual coefficients using the first quantization matrix; the decoding step includes decoding first information representing a decoding mode of the first quantization matrix from a scaling_list_data syntax structure; When the decoded first information indicates that a first mode is used, in the decoding step, second information for specifying a second quantization matrix to be referenced is decoded from the scaling_list_data syntax structure, and the first quantization matrix is derived by determining that the second quantization matrix specified based on the second information is the same as the first quantization matrix; If the decoded first information indicates that a second mode is used, the decoding step decodes the second information for identifying the second quantization matrix from the scaling_list_data syntax structure; (a) associating a first element, which is a first element in a first one-dimensional matrix consisting of a plurality of elements, with the first element being a difference value element between the element in the first row and the first column of the first quantization matrix and the element in the first row and the first column of the second quantization matrix specified based on the second information, with the first row and the first column of a two-dimensional matrix as a first element in a predetermined scanning order, and deriving the element in the first row and the first column of the first quantization matrix using the first element associated with the first row and the first column of the second quantization matrix; (b) associating a second element in the first one-dimensional matrix, the second element being a difference value element between the element in the second row, the first column of the first quantization matrix and the element in the second row, the first column of the second quantization matrix, with the second row, the first column of the two-dimensional matrix as a second element in the predetermined scanning order, and deriving the element in the second row, the first column of the first quantization matrix using the second element associated with the second row, the first column and the element in the second row, the first column of the second quantization matrix; (c) associating a third element in the first one-dimensional matrix, the third element being a difference value element between the element in the first row, second column of the first quantization matrix and the element in the first row, second column of the second quantization matrix, with the first row, second column of a two-dimensional matrix as a third element in the predetermined scanning order, and decoding the first quantization matrix by using the third element associated with the first row, second column and the element in the first row, second column of the second quantization matrix to derive the element in the first row, second column of the first quantization matrix; When the decoded first information indicates that a third mode is used, in the decoding step, a plurality of difference values including a difference value between an element in the first row and first column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order are decoded from the scaling_list_data syntax structure; (a) using a first element in a second one-dimensional matrix obtained from the plurality of difference values, deriving an element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order; (b) using a second element in the second one-dimensional matrix, deriving an element in the second row and first column of the first quantization matrix corresponding to the second element in the predetermined scanning order; (c) using a third element in the second one-dimensional matrix, deriving an element in the first row and second column of the first quantization matrix that corresponds to the third element in the predetermined scanning order; (d) decoding the first quantization matrix by using a fourth element in the second one-dimensional matrix to derive an element in the third row and first column of the first quantization matrix, which corresponds to the fourth element in the predetermined scanning order; In the decoding step, a quantization matrix in which all elements are set to 16 may be used as the second quantization matrix; The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used. If the size of the first quantization matrix is 4x4, the size of the second quantization matrix is 4x4.

1. An image decoding method comprising:

4. a quantization step of quantizing transform coefficients of a block in the image using a first quantization matrix; an encoding step of encoding the first quantization matrix, When a first mode is used to encode the first quantization matrix, in the encoding step, information indicating that the first quantization matrix is the same as a reference second quantization matrix and information for identifying the second quantization matrix are encoded into a scaling_list_data syntax structure; When a second mode is used to encode the first quantization matrix, in the encoding step, information for identifying the second quantization matrix is encoded in the scaling_list_data syntax structure; (a) a first element, which is the first element in a predetermined scanning order, and which is an element of a difference value between the element in the first row and the first column of the first quantization matrix and the element in the first row and the first column of the second quantization matrix; (b) a second element in the predetermined scanning order, the second element being a difference value element between the element in the second row and the first column of the first quantization matrix and the element in the second row and the first column of the second quantization matrix; (c) encoding information used to derive a first one-dimensional matrix consisting of a plurality of elements including a third element in the predetermined scanning order, the third element being an element of a difference value between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix; When a third mode is used to encode the first quantization matrix, the encoding step (a) encoding a difference value between the element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order and a predetermined value into the scaling_list_data syntax structure; (b) encoding a difference value between the element in the second row and the first column of the first quantization matrix corresponding to the second element in the predetermined scanning order and the element in the first row and the first column of the first quantization matrix into the scaling_list_data syntax structure; (c) encoding a difference value between the element in the first row, second column of the first quantization matrix corresponding to the third element in the predetermined scanning order and the element in the second row, first column of the first quantization matrix into the scaling_list_data syntax structure; (d) encoding a difference value between an element in the third row and first column of the first quantization matrix corresponding to the fourth element in the predetermined scanning order and an element in the first row and second column of the first quantization matrix into the scaling_list_data syntax structure; encoding a difference value between two consecutive elements in the predetermined scanning order of the first quantization matrix; In the encoding step, a quantization matrix in which all elements are set to 16 can be used as the second quantization matrix; The code amount of the information indicating that the first mode is used is smaller than the code amount of the information indicating that the second mode is used. If the size of the first quantization matrix is 4x4, the size of the second quantization matrix is 4x4.

1. An image coding method comprising:

5. A program that, when read and executed by a computer, causes the computer to execute each step of the method according to claim 3.

6. A program that, when read and executed by a computer, causes the computer to execute each step of the method according to claim 4.

Images