Image decoding apparatus, image encoding apparatus, and methods and programs thereof.

Abstract

To provide a method and a device that improve the efficiency in encoding image data by reducing the amount of encoding in a quantization matrix.SOLUTION: An image encoding device has a quantization unit that quantizes a block prediction error by using a quantization matrix, and an encoding unit that encodes the quantization matrix. The image encoding device has: a first mode for encoding information indicating that a quantization matrix to be encoded is the same as an already-encoded quantization matrix and not encoding elements of the quantization matrix to be encoded; a second mode for encoding information indicating encoding the differences between the elements of the quantization matrix to be encoded and elements of the already-encoded quantization matrix and encoding the differences between the elements of the quantization matrix to be encoded and the elements of the already-encoded quantization matrix; and a third mode for encoding information indicating encoding the differences between the elements in the quantization matrix to be encoded and encoding the differences between the elements in the quantization matrix to be encoded.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to image encoding processing and decoding processing.

Background Art

[0002] As an encoding method for compressed recording of moving images, the HEVC (High Efficiency Video Coding) encoding method (hereinafter simply referred to as HEVC) is known. In HEVC, in order to improve the encoding efficiency, a basic block having a size larger than a conventional macroblock (16×16 pixels) is adopted. This basic block having a large size is called a CTU (Coding Tree Unit), and its size is up to 64×64 pixels at maximum. A CTU is further divided into sub-blocks that are units for performing prediction and conversion.

[0003] Also, in HEVC, a process called quantization matrix is used to weight the coefficients (hereinafter referred to as orthogonal transform coefficients) after orthogonal transform according to the frequency components. By further reducing the data of high-frequency components that are less likely to be noticeable in human vision degradation, it is possible to increase the compression efficiency while maintaining the image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.

[0004] In recent years, activities for international standardization of a more efficient encoding method as a successor to HEVC have been started. The JVET (Joint Video Experts Team) was established between ISO / IEC and ITU-T, and standardization is being promoted as the VVC (Versatile Video Coding) encoding method (hereinafter referred to as VVC). In order to improve the encoding efficiency, in VVC, the size of the basic block is up to 128×128 pixels at maximum, and in addition to the conventional square sub-block division, rectangular sub-block division is also being considered.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2013-38758 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In VVC, as in HEVC, the introduction of a quantization matrix is ​​being considered. Furthermore, in VVC, a wider variety of sub-block partitions are being considered than in HEVC, including rectangular shapes. Since the distribution of orthogonal transformation coefficients corresponding to each sub-block partition differs depending on the magnitude and shape of the orthogonal transformation, it is desirable to define an optimal quantization matrix according to the size and shape of the sub-block. However, defining a separate quantization matrix for all sub-block shapes would unnecessarily increase the sign amount of the quantization matrix.

[0007] This invention has been made in view of the above problems, and aims to provide a technique for suppressing the sign amount of a quantization matrix. [Means for solving the problem]

[0008] To solve this problem, for example, the image decoding device of the present invention has the following configuration. That is, A decoding means for decoding the first quantization matrix, The system comprises an inverse quantization means that inversely quantizes the residual coefficients using the first quantization matrix, The decoding means decodes first information representing the decoding mode of the first quantization matrix from the scaling_list_data syntax structure, If the decoded first information indicates that the first mode is used, the decoding means decodes second information for identifying the second quantization matrix of reference from the scaling_list_data syntax structure, and derives the first quantization matrix by determining that the second quantization matrix identified based on the second information is the same as the first quantization matrix. If the decoded first information indicates that the second mode is used, the decoding means decodes the second information for identifying the second quantization matrix from the scaling_list_data syntax structure, (a) The first element of a first one-dimensional matrix consisting of multiple elements, which is the difference between the element in the first row and first column of the first quantization matrix and the element in the first row and first column of the second quantization matrix identified based on the second information, is associated with the first element in the first row and first column of a two-dimensional matrix as the first element in a predetermined scanning order, and the element in the first row and first column of the first quantization matrix is ​​derived using the first element associated with the first row and first column and the element in the first row and first column of the second quantization matrix, (b) The second element of the first one-dimensional matrix, which is the difference between the element in the second row and first column of the first quantization matrix and the element in the second row and first column of the second quantization matrix, is associated with the second element in the second row and first column of the two-dimensional matrix as the second element in the predetermined scanning order, and the element in the second row and first column of the first quantization matrix is ​​derived using the second element associated with the second row and first column and the element in the second row and first column of the second quantization matrix. (c) The third element of the first one-dimensional matrix, which is the difference between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix, is associated with the first row and second column of the two-dimensional matrix as the third element in the predetermined scanning order, and the first quantization matrix is ​​decoded by deriving the element in the first row and second column of the first quantization matrix using the third element associated with the first row and second column and the element in the first row and second column of the second quantization matrix. If the decoded first information indicates that the third mode is used, the decoding means decodes a plurality of difference values ​​from the scaling_list_data syntax structure, including the difference between the element in the first row and first column of the first quantization matrix and a predetermined value, and the difference between two consecutive elements in the predetermined scanning order of the first quantization matrix. (a) Using the first element of the second one-dimensional matrix obtained from the plurality of difference values, derive the element that is the first row and first column of the first quantization matrix, which corresponds to the first in the predetermined scanning order, (b) Using the second element of the second one-dimensional matrix, derive the element that is the second row and first column of the first quantization matrix, which corresponds to the second in the predetermined scan order. (c) Using the third element of the second one-dimensional matrix, derive the element that corresponds to the third in the predetermined scanning order, which is the first row and second column of the first quantization matrix, (d) The first quantization matrix is ​​decoded by using the fourth element of the second one-dimensional matrix to derive the element that is in the third row and first column of the first quantization matrix, which corresponds to the fourth element of the predetermined scanning order. The decoding means can use a quantization matrix in which all elements are set to 16 as the second quantization matrix. The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used. The size of the second quantization matrix is ​​limited to one of the following sizes: 2x2, 4x4, or 8x8. The 2x2 size second quantization matrix is ​​applicable to chroma blocks, but not to luma blocks. When decoding the 2x2 size second quantization matrix, the 4x4 size second quantization matrix, and the 8x8 size second quantization matrix, the 2x2 size second quantization matrix is ​​decoded first. The size of the first quantization matrix is ​​4 × In the case of 4, the second quantization matrix is ​​4 × It is 4 An image decoding device characterized by the following features. [Effects of the Invention]

[0009] According to the present invention, it becomes possible to suppress the code amount of the quantization matrix compared to conventional methods, thereby improving the encoding efficiency of image data. [Brief explanation of the drawing]

[0010] [Figure 1] Block diagram of the image encoding device according to the first embodiment. [Figure 2]Block configuration diagram of the image decoding device according to the first embodiment. [Figure 3] Flowchart showing the encoding process in the image encoding device according to the first embodiment. [Figure 4] Flowchart showing the decoding process in the image decoding device according to the first embodiment. [Figure 5] Computer hardware configuration diagram applied in the second embodiment. [Figure 6] Diagram showing an example of the data structure of the bitstream generated by the image encoding device according to the first embodiment. [Figure 7] Diagram showing an example of sub-block division in the embodiment. [Figure 8] Diagram showing an example of the quantization matrix used in the embodiment. [Figure 9] Diagram showing the scanning method of the elements of the quantization matrix used in the embodiment. [Figure 10] Diagram showing the difference value matrix of the quantization matrix generated in the embodiment. [Figure 11] Diagram showing an example of the encoding table used for encoding the difference values of the quantization matrix. [Figure 12] Diagram showing an example of the syntax table of the quantization matrix used in the embodiment. [Figure 13] Diagram showing an example of the reference quantization matrix used in the embodiment. [Figure 14] Flowchart showing the encoding procedure of the quantization matrix.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0012] [First Embodiment] Although the embodiments describe several features, not all of these features are necessarily essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, the same reference numeral is used for identical or similar configurations, and redundant descriptions are omitted.

[0013] Figure 1 is a block diagram of an image encoding device in the first embodiment. The image encoding device has a control unit 150 that controls the entire device. This control unit 150 has a CPU, a ROM that stores the program executed by the CPU, and RAM that is used as the CPU's work area. The image encoding device also has an input terminal 101, a block division unit 102, a quantization matrix holding unit 103, a prediction unit 104, a conversion / quantization unit 105, an inverse quantization / inverse conversion unit 106, an image playback unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0014] The input terminal 101 receives the image data to be encoded, generated by the image data source, in frame units. The image data source can be of any type, such as an imaging device, a file server or storage medium storing the image data to be encoded. The output terminal 112 outputs the encoded data to an output destination device, which can also be a storage medium, file server, or other device.

[0015] The block division unit 102 divides the input frame image into multiple basic blocks and outputs one of them as a basic block to the subsequent prediction unit 104 in sequence.

[0016] The quantization matrix holding unit 103 generates multiple quantization matrices prior to encoding and stores them in an internal memory (not shown). The method for generating the quantization matrices is not particularly limited; the user may input the quantization matrix, calculate it from the characteristics of the input image, or use a pre-specified initial value. In this embodiment, the quantization matrix holding unit 103 generates and stores two-dimensional quantization matrices 800 to 811 corresponding to orthogonal transformations of 8x8 pixel size, 4x4 pixel size, or 2x2 pixel size shown in Figures 8(a) to (l). Here, quantization matrices 800, 803, 806, and 809 are for the luminance component, and quantization matrices 801, 802, 804, 805, 807, 808, 810, and 811 are for the two color difference components.

[0017] The prediction unit 104 determines sub-block division for image data in basic block units, and generates predicted image data by performing intra-prediction (intra-frame prediction) and inter-prediction (inter-frame prediction) on a sub-block basis. Furthermore, the prediction unit 104 calculates and outputs the prediction error at the pixel level from the sub-blocks in the input image data and the corresponding predicted image data. The prediction unit 104 also outputs information necessary for prediction, such as sub-block division, prediction mode, and motion vector, along with the prediction error. Hereafter, this information necessary for prediction will be referred to as prediction information.

[0018] The transformation and quantization unit 105 orthogonally transforms the prediction error of the subblock input from the prediction unit 104 on a subblock-by-subblock basis to obtain orthogonal transformation coefficients. Furthermore, the transformation and quantization unit 105 uses the quantization matrix stored in the quantization matrix holding unit 103 to quantize the orthogonal transformation coefficients and obtain residual coefficients (orthogonal transformation coefficients after quantization).

[0019] The inverse quantization / inverse transformation unit 106 receives residual coefficients from the transformation / quantization unit 105, performs inverse quantization using the corresponding quantization matrix stored in the quantization matrix holding unit 103, and reconstructs the orthogonal transformation coefficients. The inverse quantization / inverse transformation unit 106 further performs an inverse orthogonal transformation on the orthogonal transformation coefficients to reconstruct the prediction error.

[0020] The image playback unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image playback unit 107 generates reproduced image data by adding the prediction error input from the inverse quantization / inverse transform unit 106 to this predicted image data and stores it in the frame memory 108.

[0021] The in-loop filter 109 performs in-loop filtering, such as deblocking filtering and sample adaptive offsetting, on the playback image stored in the frame memory 108, and stores the filtered image data back into the frame memory 108.

[0022] The encoding unit 110 encodes the residual coefficients output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104 to generate coded data, which is then output to the integrated encoding unit 111.

[0023] The quantization matrix coding unit 113 encodes the quantization matrix held in the quantization matrix holding unit 103 (see Figures 8(a) to (l)) to generate quantization matrix coded data, and outputs it to the integrated coding unit 111.

[0024] The integrated coding unit 111 generates header code data, which includes quantization matrix code data from the quantization matrix coding unit 113. The integrated coding unit 111 then adds the code data output from the coding unit 110 to the header code data to form a bitstream. The integrated coding unit 111 then outputs the formed bitstream via the output terminal 112. The image encoding operation in the image encoding device will now be explained in more detail. In this embodiment, the input terminal 101 is configured to input moving image data in a 4:2:0 color format in frame units at a predetermined frame rate (for example, 30 frames / second), but it is also acceptable to configure it to input still image data for one frame. Furthermore, in this embodiment, for the sake of explanation, the block division unit 101 will be described as dividing the image data input from the input terminal 101 into 8x8 pixel basic blocks. That is, an 8x8 pixel basic block will contain pixels for the luminance (Y) component of 8x8 pixels and pixels for the color difference (Cb and Cr) component of 4x4 pixels. Note that this is for the sake of ease of understanding and is not limited to the above numerical values ​​(sizes).

[0025] Prior to image encoding, the quantization matrix is ​​generated and encoded.

[0026] The quantization matrix holding unit 103 first generates and holds a quantization matrix. Specifically, it generates a quantization matrix according to the size of the subblock to be encoded and the type of prediction method. In this embodiment, it generates a quantization matrix corresponding to an 8x8 pixel basic block that is not divided into subblocks as shown in Figure 7(a), and a quantization matrix corresponding to a 4x4 pixel subblock obtained by quadtree-dividing the basic block as shown in Figure 7(b). That is, the quantization matrix holding unit 103 generates 8x8 pixel size and 4x4 pixel size quantization matrices for the luminance (Y) component, and 4x4 pixel size and 2x2 pixel size quantization matrices for the chrominance (Cb and Cr) component. However, the generated quantization matrices are not limited to these, and quantization matrices corresponding to the shape of the subblock, such as 4x8 and 8x4, may be generated. The method for determining each element constituting the quantization matrix is ​​not particularly limited. For example, predetermined initial values ​​may be used, or they may be set individually. They may also be generated according to the characteristics of the image.

[0027] The quantization matrix holding unit 103 holds the multiple types of quantization matrices generated in this way in an internal memory (not shown). Figure 8(a) shows an 8x8 pixel size quantization matrix 800 for the Y component using intra-prediction. Figure 8(b) shows a 4x4 pixel size quantization matrix 801 for the Cb component using intra-prediction, and Figure 8(c) shows a 4x4 pixel size quantization matrix 802 for the Cr component using intra-prediction.

[0028] Similarly, Figure 8(d) shows an 8x8 pixel quantization matrix 803 for the Y component using interpretation. Figure 8(e) shows a 4x4 pixel quantization matrix 804 for the Cb component using interpretation, and Figure 8(f) shows a 4x4 pixel quantization matrix 805 for the Cr component using interpretation.

[0029] Furthermore, Figure 8(g) shows a 4x4 pixel size quantization matrix 806 for the Y component using intra-prediction. Also, Figure 8(h) shows a 2x2 pixel size quantization matrix 807 for the Cb component using intra-prediction, and Figure 8(i) shows a 2x2 pixel size quantization matrix 808 for the Cr component using intra-prediction.

[0030] Similarly, Figure 8(j) shows a 4x4 pixel size quantization matrix 809 for the Y component using interpretation. Also, Figure 8(k) shows a 2x2 pixel size quantization matrix 810 for the Cb component using interpretation, and Figure 8(l) shows a 2x2 pixel size quantization matrix 811 for the Cr component using interpretation.

[0031] To simplify the explanation, the configuration will consist of 64 pixels in an 8x8 grid, 16 pixels in a 4x4 grid, and 4 pixels in a 2x2 grid, with each square within the thick border representing an element constituting the quantization matrix. In this embodiment, the 12 types of quantization matrices shown in Figures 8(a) to 8(l) are assumed to be held in a two-dimensional shape, but the elements within the quantization matrix are not limited to these. Furthermore, depending on the size of the subblocks, it is possible to hold multiple quantization matrices for the same color component and the same prediction mode. Generally, to realize quantization processing according to human visual characteristics, the elements in the low-frequency portion corresponding to the upper left part of the quantization matrix are small, and the elements in the high-frequency portion corresponding to the lower right part are large, as shown in Figures 8(a) to 8(l).

[0032] The quantization matrix coding unit 113 sequentially reads out the two-dimensional quantization matrices held in the quantization matrix holding unit 106 and determines the coding mode for each quantization matrix to be used when coding it. In this embodiment, three types of quantization matrix coding modes are used: "quantization matrix reference mode," "inter-quantization matrix difference coding mode," and "intra-quantization matrix difference coding mode." For each quantization matrix, the quantization matrix coding unit 113 determines the quantization matrix coding mode that minimizes the amount of data in the coding result.

[0033] Here, we will explain in more detail the three types of quantization matrix coding modes used in the quantization matrix coding unit 113 of this embodiment.

[0034] The "quantization matrix reference mode" is used when there is a match between the quantization matrix to be encoded and the encoded quantization matrix among the encoded quantization matrices. When this mode is used, the quantization matrix encoding unit 113 encodes a quantization matrix reference index, which is an identifier that indicates the matching encoded quantization matrix to be referenced, instead of encoding each element of the quantization matrix to be encoded.

[0035] Next, the "quantization matrix difference coding mode" is a mode in which a quantization matrix that is close to the target quantization matrix is ​​selected from the coded quantization matrices, the difference value of each element is calculated, and the calculated difference value is coded. When this mode is selected, the quantization matrix coding unit 113 codes a quantization matrix reference index, which is an identifier indicating the quantization matrix to be referenced, and then codes difference information, which is a set of difference values ​​for each element.

[0036] The "quantization matrix differential coding mode" is used when no coded quantization matrix exists or when there is no quantization matrix similar to the one to be coded. The quantization matrix coding unit scans each element of the quantization matrix to be coded, calculates the difference values ​​between elements, and codes the difference information, which is the set of difference values.

[0037] Figure 12 is the syntax table used for encoding the quantization matrix in this embodiment. Using this syntax table, the encoding process for each quantization matrix shown in Figures 8(a) to 8(l) in this embodiment will be explained in detail. First, the structure of the syntax table in Figure 12 will be explained. At the outermost part of the syntax table is a for loop with a parameter called sizeId. sizeId indicates the size of the quantization matrix, and quantization matrices of the same size will be encoded consecutively. sizeId=1 indicates a 2x2 pixel size quantization matrix. Similarly, sizeId=2 indicates a 4x4 pixel size quantization matrix, and sizeId=3 indicates an 8x8 pixel size quantization matrix.

[0038] The initial value of sizeId in the for loop is "1", and it is defined that it increases by "1" with each loop iteration, and the loop continues as long as sizeId is less than 4. Therefore, in this embodiment, it means that the quantization matrix group of 2x2 pixels is encoded first, followed by the quantization matrix group of 4x4 pixels, and then the quantization matrix group of 8x8 pixels, and so on. VVC uses orthogonal transformations ranging in size from a minimum of 2x2 pixels to 64x64 pixels, and it is also possible to configure the for loop to go from a 2x2 pixel size with sizeId=1 to a 64x64 pixel size with sizeId=6, encoding the corresponding quantization matrix for each. However, in this embodiment, the maximum orthogonal transformation size, i.e., the size of the subblock, is 8x8, so the upper limit of the for loop is set to sizeId=3, which corresponds to an 8x8 pixel size. In this way, by setting the upper limit of the for loop based on the maximum value of the orthogonal transformation size actually used, it is possible to omit the encoding of unnecessary quantization matrices and prevent the generation of redundant codes.

[0039] Inside the for loop based on sizeId, there is another for loop based on a parameter called matrixId. This matrixId is a parameter based on the prediction mode and color component. A value of 0 means it is for intra-prediction (Y component), 1 means it is for intra-prediction (Cb component), and 2 means it is for intra-prediction (Cr component). Similarly, a matrixId of 3 means it is for inter-prediction (Y component), 4 means it is for inter-prediction (Cb component), and 5 means it is for inter-prediction (Cr component). In other words, within a group of quantization matrices of the same size, the group of quantization matrices corresponding to intra-prediction is encoded first, followed by the group of quantization matrices corresponding to inter-prediction. Furthermore, within a group of quantization matrices corresponding to the same prediction mode, they are encoded in the order of Y component, Cb component, and Cr component.

[0040] Based on the coding order of the quantization matrices described above, the coding order of each quantization matrix in Figures 8(a) to (l) is as follows:

[0041] First, the 2x2 pixel figures 8(h), (i), (k), and (l) are encoded. Next, the 4x4 pixel figures 8(g), (b), (c), (j), (e), and (f) are encoded. Finally, the 8x8 pixel figures 8(a) and (d) are encoded.

[0042] The encoding order described above essentially follows this priority order: (i) the size of the quantization matrix, (ii) the type for intra / internet use, and (iii) the type of component (luminance, chrominance). The quantization matrix size is given the highest priority because the encoding takes into account the correlation between quantization matrices.

[0043] Next, the coding procedure for each quantization matrix will be described. First, the quantization matrix coding unit 113 encodes the determined quantization matrix coding mode. In the syntax table of Figure 12, scaling_list_pred_mode_idx corresponds to the quantization matrix coding mode. In this embodiment, when this value is 0, the "quantization matrix reference mode" is used; when it is 1, the "inter-quantization matrix difference coding mode" is used; and when it is 2, the "intra-quantization matrix difference coding mode" is used. Since the quantization matrix coding mode is Golomm coded, the smaller this value is, the smaller the amount of code related to scaling_list_pred_mode_idx. That is, in this embodiment, the amount of code related to the matrix reference mode is the smallest, which has the advantage of minimizing the amount of code generated when there are many identical quantization matrices.

[0044] When using the quantization matrix reference mode, the quantization matrix encoding unit 113 then encodes the quantization matrix reference index. This corresponds to scaling_list_pred_matrix_id_delta in the syntax table in Figure 12. For example, the quantization matrix 801 in Figure 8(b) is identical to the quantization matrix 806 in Figure 8(g), which is encoded immediately before it. Therefore, when encoding the quantization matrix 801 in Figure 8(b), the quantization matrix encoding unit 113 selects the quantization matrix reference mode and encodes 0 as the quantization matrix reference index to indicate that it is the quantization matrix encoded immediately before it. In this way, it is possible to encode a quantization matrix identical to the quantization matrix 806 in Figure 8(g). If the previous quantization matrix is ​​used, the quantization matrix reference index will be "1".

[0045] On the other hand, when using the inter-quantization matrix difference coding mode, similar to when using the quantization matrix reference mode, the quantization matrix coding unit 113 first encodes the quantization matrix reference index. Subsequently, the quantization matrix coding unit 113 encodes difference information, which is the difference between each element of the encoded reference quantization matrix determined by the quantization matrix reference index and each element of the quantization matrix to be encoded. This corresponds to scaling_list_pred_delta in the syntax table of Figure 12. For example, each element of the quantization matrix 802 in Figure 8(c) is similar to each element of the quantization matrix 801 in Figure 8(b) that is encoded immediately before (in this example, the absolute value of the difference between each element is less than or equal to a predetermined value (less than or equal to "1" in the figure)). Therefore, when encoding the quantization matrix 802 in Figure 8(c), the quantization matrix coding unit 113 selects the inter-quantization matrix difference coding mode and first encodes 0 as the quantization matrix reference index to indicate that it is the quantization matrix that was encoded immediately before. Furthermore, when encoding the difference between each element of the previous quantization matrix, the quantization matrix reference index becomes "1". Next, the quantization matrix encoding unit 113 calculates the difference between each element of the reference quantization matrix in Figure 8(b) and each element of the quantization matrix to be encoded in Figure 8(c), and calculates the two-dimensional difference value matrix shown in Figure 10(a). Then, it scans each difference value of the two-dimensional difference value matrix and places it into a one-dimensional matrix. Figures 9(a) to 9(c) show the scanning method used when a two-dimensional matrix is ​​placed into one dimension in this embodiment, with Figure 9(a) corresponding to an 8×8, Figure 9(b) to a 4×4, and Figure 9(c) to a 2×2 quantization matrix. Since a 4×4 quantization matrix is ​​the target here, the quantization matrix encoding unit 113 uses the scanning method shown in Figure 9(b) to place the two-dimensional difference value matrix in Figure 10(a) into the one-dimensional difference value matrix shown in Figure 10(b). The quantization matrix coding unit 113 then encodes each difference value in Figure 10(b). Note that the absolute value of each difference value in Figure 10(b) is "1" or less, but there may be cases where the absolute value is greater than 1.By taking such cases into consideration and further obtaining the difference between adjacent difference values, it becomes possible to reduce the amount of sign.

[0046] Furthermore, when using the intra-quantization matrix difference coding mode, the quantization matrix coding unit 113 scans each element of the quantization matrix to be coded using one of the methods shown in Figures 9(a) to (c), calculates the difference between two consecutive elements, and places them in a one-dimensional matrix. The quantization matrix coding unit 113 then codes each difference value placed in the one-dimensional matrix as difference information. This corresponds to scaling_list_delta_coef in the syntax table of Figure 12. For example, the quantization matrix 806 in Figure 8(g) is the first to be coded in a 4x4 pixel size quantization matrix, so there is no quantization matrix to be referenced using the aforementioned quantization matrix reference mode or inter-quantization matrix difference coding mode. Therefore, when coding the quantization matrix 806 in Figure 8(g), the quantization matrix coding unit 113 inevitably selects the intra-quantization matrix difference coding mode. Therefore, the quantization matrix coding unit 113 scans the quantization matrix 806 in Figure 8(g) using Figure 9(b) and calculates the difference between each element and the previous element in the scanning order. The quantization matrix coding unit 113 arranges the calculated differences in a one-dimensional matrix to obtain the one-dimensional difference value matrix shown in Figure 10(c). Here, for example, the 4x4 pixel size quantization matrix in Figure 8(g) is scanned by the scanning method shown in Figure 9(b), but after the first element 6 located in the upper left, the element 13 located immediately below it is scanned, and the difference of +7 is calculated. Also, the encoding of the first element of the quantization matrix (6 in this embodiment) is calculated by calculating the difference with a predetermined initial value (for example, 8), but of course it is not limited to this, and the difference with any value or the value of the first element itself may be used. In short, it is sufficient that it is the same initial value as the decoding device. Then, the quantization matrix coding unit 113 encodes each difference value in Figure 10(c).

[0047] Here, we will provide a supplementary explanation regarding the encoding process of the quantization matrix described above. The fifth item from the top in Figure 12, “scaling_list_pred_mode_idx[sizeId][matrixId]”, is a parameter that indicates the encoding mode of the quantization matrix according to sizeId and matrixId. To calculate this parameter, the quantization matrix specified by sizeId and matrixId (one of Figures 8(a) to (l)) is encoded according to the three encoding modes described above, and the encoding mode that produces the smallest amount of code is determined. Then, information that identifies the determined encoding mode (0 to 2 in the above example) is set in this parameter.

[0048] The quantization matrix coding unit 113 generates quantization matrix coded data as the coding result of each quantization matrix generated using each quantization matrix coding mode as described above. In this embodiment, the quantization matrix coding unit 113 generates quantization matrix coded data by coding each element of the one-dimensional difference matrix with a codeword, which is a binary code, using the coding table shown in Figure 11(a). Note that the coding table is not limited to this, and for example, the coding table shown in Figure 11(b) may be used. In this way, the quantization matrix coding unit 113 outputs the generated quantization matrix coded data to the subsequent integrated coding unit 111.

[0049] Returning to Figure 1, the integrated encoding unit 111 integrates the coded data of the quantization matrix with the header information necessary for encoding the image data.

[0050] Next, the image data is encoded. The image data for one frame input from the input terminal 101 is supplied to the block division unit 102.

[0051] The block division unit 102 divides the input 1-frame image data into multiple basic blocks and outputs the image data for each basic block to the prediction unit 104. In this embodiment, image data for each 8x8 pixel basic block is supplied to the prediction unit 104.

[0052] The prediction unit 104 performs prediction processing on image data in basic block units input from the block division unit 102. Specifically, it determines subblock division, which further divides the basic block into smaller subblocks, and then determines a prediction mode such as intra-prediction or inter-prediction for each subblock. Intra-prediction generates prediction pixels for the target block using pre-coded pixels located spatially around the target block, and also generates an intra-prediction mode indicating an intra-prediction method such as horizontal prediction, vertical prediction, or DC prediction. Inter-prediction generates prediction pixels for the target block using pre-coded pixels from a frame that is temporally different from the target block, and also generates motion information indicating the reference frame and motion vector.

[0053] The subblock division method will be explained with reference to Figure 7. In Figures 7(a) to (f), the thick borders of blocks 700 to 705 represent the same 8x8 pixel size as the basic blocks. Each rectangle within the thick border represents a subblock. Figure 7(b) shows an example of conventional square subblock division, where the 8x8 pixel basic block 701 is divided into four 4x4 pixel subblocks. On the other hand, Figures 7(c) to (f) show examples of rectangular subblock division. Figure 7(c) shows that the basic block 702 is divided into two 4x8 pixel subblocks (longer vertically). Figure 7(d) shows that the basic block 703 is divided into two 8x4 pixel subblocks (longer horizontally). In Figures 7(e) and (f), the basic blocks 704 and 705 are divided into three rectangular subblocks in a 1:2:1 ratio, although the division method is different. Thus, encoding processing is performed using not only square subblocks but also rectangular subblocks.

[0054] In this embodiment, we will explain using either Figure 7(a), in which the 8x8 pixel basic block is not divided into subblocks, or Figure 7(b), in which it is divided into subblocks into a quadtree. However, the method of subblock division is not limited to these. A ternary tree division as shown in Figures 7(e) and (f), or a binary tree division as shown in Figures 7(c) and 7(d) may also be used. If a subblock division other than those shown in Figures 7(a) and 7(b) is used, a quantization matrix corresponding to the subblock used will be generated in the quantization matrix holding unit 103. The generated quantization matrix will then be encoded in the quantization matrix encoding unit 113.

[0055] The prediction unit 104 generates predicted image data from the determined prediction mode and the encoded region stored in the frame memory 108. It also calculates a pixel-level prediction error from the predicted image data corresponding to the subblock of interest in the input image data and outputs this error to the conversion / quantization unit 105. Furthermore, the prediction unit 104 outputs information such as subblock division and prediction mode as prediction information to the encoding unit 110 and the image playback unit 107.

[0056] The transformation and quantization unit 105 performs orthogonal transformation and quantization on the prediction error input from the prediction unit 104 to generate residual coefficients. Specifically, the transformation and quantization unit 105 first applies an orthogonal transformation process to the prediction error corresponding to the size of the subblock to generate orthogonal transformation coefficients. Then, the transformation and quantization unit 105 quantizes the orthogonal transformation coefficients using the quantization matrix stored in the quantization matrix holding unit 103 according to the prediction mode and color component to generate residual coefficients. In this embodiment, when the intra-prediction mode is used without subblock division, the quantization matrix shown in Figure 8(a) is used for the orthogonal transformation coefficient of the Y component, Figure 8(b) for the orthogonal transformation coefficient of the Cb component, and Figure 8(c) for the orthogonal transformation coefficient of the Cr component. Similarly, when the inter-prediction mode is used instead of subblock division, the quantization matrix shown in Figure 8(d) is used for the orthogonal transformation coefficient of the Y component, Figure 8(e) for the orthogonal transformation coefficient of the Cb component, and Figure 8(f) for the orthogonal transformation coefficient of the Cr component. On the other hand, when the subblock division shown in Figure 7(b) is performed and the intra-prediction mode is used, the quantization matrix shown in Figure 8(g) is used for the orthogonal transformation coefficients of the Y component, Figure 8(h) for the orthogonal transformation coefficients of the Cb component, and Figure 8(i) for the orthogonal transformation coefficients of the Cr component. Similarly, when the subblock division shown in Figure 7(b) is performed but the inter-prediction mode is used instead, the quantization matrix shown in Figure 8(j) is used for the orthogonal transformation coefficients of the Y component, Figure 8(k) for the orthogonal transformation coefficients of the Cb component, and Figure 8(l) for the orthogonal transformation coefficients of the Cr component is used. However, the quantization matrix used is not limited to these. The generated residual coefficients and chromatic difference integration information are output to the encoding unit 110 and the inverse quantization / inverse transformation unit 106.

[0057] The inverse quantization / inverse transformation unit 106 reconstructs the orthogonal transformation coefficients by inverse quantizing the residual coefficients input from the transformation / quantization unit 105 using the corresponding quantization matrix stored in the quantization matrix holding unit 103. The inverse quantization / inverse transformation unit 106 further reconstructs the prediction error by inverse orthogonal transforming the reconstructed orthogonal transformation coefficients. Similar to the transformation / quantization unit 105, the inverse quantization process uses quantization matrices corresponding to the size of the subblock to be encoded and the color components. Specifically, the inverse quantization / inverse transformation unit 106 performs inverse quantization using the same quantization matrix as used in the transformation / quantization unit 105. That is, when using intra-prediction mode without subblock division, the quantization matrices shown in Figure 8(a) are used for the orthogonal transformation coefficients of the Y component, Figure 8(b) for the orthogonal transformation coefficients of the Cb component, and Figure 8(c) for the orthogonal transformation coefficients of the Cr component. Similarly, when subblock partitioning is not performed and interprediction mode is used instead, the quantization matrices shown in Figure 8(d) are used for the orthogonal transformation coefficients of the Y component, Figure 8(e) for the orthogonal transformation coefficients of the Cb component, and Figure 8(f) for the orthogonal transformation coefficients of the Cr component. On the other hand, when subblock partitioning as shown in Figure 7(b) is performed and intraprediction mode is used, the quantization matrices shown in Figure 8(g) are used for the orthogonal transformation coefficients of the Y component, Figure 8(h) for the orthogonal transformation coefficients of the Cb component, and Figure 8(i) for the orthogonal transformation coefficients of the Cr component are used. Similarly, when subblock partitioning as shown in Figure 7(b) is performed and interprediction mode is used instead, the quantization matrices shown in Figure 8(j) are used for the orthogonal transformation coefficients of the Y component, Figure 8(k) for the orthogonal transformation coefficients of the Cb component, and Figure 8(l) for the orthogonal transformation coefficients of the Cr component are used.

[0058] The reconstructed orthogonal transformation coefficients are then subjected to an inverse orthogonal transformation, and the resulting reconstructed prediction error is output to the image reconstruction unit 107.

[0059] The image playback unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104 and plays back the predicted image. Then, the image playback unit 107 generates playback image data for each subblock based on the played-back predicted image and the prediction error of the subblock played back by the inverse quantization / inverse transform unit 106, and stores it in the frame memory 108.

[0060] The in-loop filter unit 109 reads the playback image data from the frame memory 108 and performs in-loop filtering, such as deblocking filtering. Then, the in-loop filter unit 109 re-stores the filtered image data back into the frame memory 108.

[0061] The encoding unit 110 entropy encodes the residual coefficients and chromatic difference integration information of the subblock units generated by the transformation / quantization unit 105, as well as the prediction information input from the prediction unit 104, to generate coded data. The method of entropy encoding is not specifically defined, but Golomb coding, arithmetic coding, Huffman coding, etc., can be used. The encoding unit 110 outputs the generated coded data to the integrated encoding unit 111.

[0062] The integrated encoding unit 111 multiplexes the encoded data input from the encoding unit 110 along with the encoded data of the header mentioned above to form a bitstream. The integrated encoding unit 111 then outputs the formed bitstream to an external source (such as a storage medium or network) from the output terminal 112.

[0063] Figure 6(a) shows an example of the data structure of the bitstream output in this embodiment. The sequence header contains the coded data of the quantization matrix and is composed of the coded results of each quantization matrix. However, the location where the coded data of the quantization matrix is ​​stored is not limited to this, and it is also possible to have it placed in a picture header section or a header section that spans multiple pictures, as shown in Figure 6(b). Furthermore, when changing the quantization matrix within a single sequence, it is possible to update it by newly coding the quantization matrix. In this case, all quantization matrices may be rewritten, or it is possible to change only a part of them by specifying the size, prediction mode, and color components of the quantization matrix corresponding to the quantization matrix to be rewritten.

[0064] Figure 3 is a flowchart showing the encoding process for one frame by the control unit 150 in the image encoding device of the embodiment.

[0065] First, prior to image encoding, in S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold a two-dimensional quantization matrix. In this embodiment, the quantization matrix holding unit 103 corresponds to blocks of 8x8 to 2x2 pixel size and generates and holds quantization matrices corresponding to each color component and prediction mode shown in Figures 8(a) to 8(l).

[0066] In S302, the control unit 150 controls the quantization matrix coding unit 113 to encode the quantization matrix generated and held in S301. The specific operation of the quantization matrix coding unit 113 here has already been explained and will be omitted. In this embodiment, the control unit 150 controls the quantization matrix coding unit 113 to encode the quantization matrices 801 to 812 shown in Figures 8(a) to (l) based on the syntax table in Figure 12, thereby generating quantization matrix coded data.

[0067] In S303, the control unit 150 controls the integrated encoding unit 111 to encode the header information necessary for encoding the image data along with the generated quantization matrix code data, and output it.

[0068] In S304, the control unit 150 controls the block division unit 102 to divide the input image in frame units into basic block units.

[0069] In S305, the control unit 150 controls the prediction unit 104 to divide the basic block image data generated in S304 into subblocks and perform prediction processing for each subblock, thereby generating prediction information such as subblock division information and prediction mode, as well as predicted image data. Furthermore, the control unit 150 controls the prediction unit 104 to calculate the prediction error from the input subblock image data and predicted image data.

[0070] In S306, the control unit 150 controls the transformation / quantization unit 105 to perform an orthogonal transformation on the prediction error calculated in S305 and generate orthogonal transformation coefficients. Furthermore, the control unit 150 controls the transformation / quantization unit 105 to perform quantization using the quantization matrix generated and held in S301 and generate residual coefficients. In this embodiment, the quantization matrices shown in Figures 8(a) to 8(l) are used depending on the size of the subblock, the prediction mode, and the color components.

[0071] In S307, the control unit 150 controls the inverse quantization / inverse transformation unit 106 to perform inverse quantization on the residual coefficients generated in S306 using the quantization matrix generated and held in S301, thereby reconstructing the orthogonal transformation coefficients. In this step, the same quantization matrix used in S306 is used for the inverse quantization process. Then, an inverse orthogonal transformation is performed on the reconstructed orthogonal transformation coefficients to reconstruct the prediction error.

[0072] In S308, the control unit 150 controls the image playback unit 107 to play back a predicted image based on the prediction information generated in S305, and then plays back image data from the played-back predicted image and the prediction error generated in S307, and stores it in the frame memory 108.

[0073] In S309, the control unit 150 controls the coding unit 110 to encode the prediction information generated in S305 and the residual coefficients generated in S306, thereby generating coded data. The coding unit 110 also outputs the generated coded data to the integrated coding unit 111. The integrated coding unit 111 positions the coded data from the coding unit 110 so as to follow the previously generated header and outputs it.

[0074] In S310, the control unit 150 determines whether the encoding of all basic blocks in the frame of interest has been completed. If the control unit 150 determines that it has been completed, it proceeds to S311. If it determines that there are still unencoded basic blocks remaining, it returns to S304 and continues encoding the next basic block.

[0075] In S311, the control unit 150 controls the in-loop filter unit 109 to perform in-loop filtering on the image data reproduced in S308, generate a filtered image, and terminate the process.

[0076] With the above configuration and operation, particularly in S302, the amount of code generated by the quantization matrix can be suppressed by determining the quantization matrix coding mode that minimizes the amount of code generated by the quantization matrix and then coding.

[0077] In this embodiment, only quantization matrices of the same size can be referenced in the quantization matrix reference mode and the inter-quantization matrix difference coding mode. However, it is also possible to configure the system to reference quantization matrices of different sizes using scaling. For example, when encoding the quantization matrix 806 in Figure 8(g), it is possible to reference the quantization matrix 807 in Figure 8(h) to scale it up, and then encode the difference between the two as difference information. This makes it possible to further reduce the amount of data in the quantization matrix that is initially encoded for each size.

[0078] Furthermore, in this embodiment, as shown in the syntax table in Figure 12, the encoding is performed in order from the smallest quantization matrix to the largest. However, it is also possible to encode the data in order from the largest quantization matrix to the largest. This makes it possible to further reduce the amount of data in the quantization matrix, especially when referencing between quantization matrices of different sizes is enabled.

[0079] Furthermore, in this embodiment, the reference target of the quantization matrix is ​​limited to an encoded quantization matrix, but as shown in Figures 13(a) to (c), a flat quantization matrix that serves as a reference when a quantization matrix is ​​not used can also be used as the reference target. This makes it possible to further reduce the amount of data in the quantization matrix, especially when the quantization matrix to be encoded is close to a flat quantization matrix.

[0080] Figure 2 is a block diagram of the image decoding device that decodes the encoded image data generated by the image encoding device described above. The configuration and operation of the decoding process will be explained below with reference to this figure.

[0081] The image decoding device has a control unit 250 that controls the entire device. This control unit 250 has a CPU, a ROM that stores the program executed by the CPU, and RAM that is used as the CPU's work area. The image decoding device also has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization / inverse transformation unit 204, an image playback unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0082] Input terminal 201 is for inputting an encoded bitstream. The input source can be, for example, a storage medium containing the encoded stream, but it can also be input from a network, and the type of source is not specified.

[0083] The decoupling and decoding unit 202 separates the bitstream into coded data related to the decoding process and coefficients, and also decodes the coded data present in the header portion of the bitstream. In this embodiment, the decoupling and decoding unit 202 separates the quantization matrix coded data and outputs it to the quantization matrix decoding unit 209. The decoupling and decoding unit 202 also outputs the coded data of the image to the decoding unit 203. In other words, the decoupling and decoding unit 202 operates in the reverse direction of the integrated encoding unit 111 in Figure 1.

[0084] The quantization matrix decoding unit 209 reconstructs and maintains the quantization matrix by decoding the quantization matrix code data supplied from the separation decoding unit 202.

[0085] The decoding unit 203 decodes the coded data of the image output from the separation decoding unit 202 and reconstructs the residual coefficients and prediction information for each subblock.

[0086] The inverse quantization / inverse transformation unit 204, similar to the inverse quantization / inverse transformation unit 106 in Figure 1, uses the reconstructed quantization matrix to perform inverse quantization on the residual coefficients of the subblock of interest, obtains the coefficients after inverse quantization, and then performs an inverse orthogonal transformation to reconstruct the prediction error.

[0087] The image playback unit 205 generates predicted image data by appropriately referring to the frame memory 206 based on the input prediction information. Then, the image playback unit 205 generates the playback image data of the subblock of interest from this predicted image data and the prediction error reproduced by the inverse quantization / inverse transform unit 204, and stores it in the frame memory 206.

[0088] The in-loop filter unit 207, similar to the in-loop filter unit 109 in Figure 1, performs in-loop filtering, such as a deblocking filter, on the regenerated image data stored in the frame memory 206, and then re-stores the filtered image data back into the frame memory 206.

[0089] Output terminal 208 sequentially outputs frame images stored in frame memory 206 to an external device. The output destination is typically a display device, but other devices may also be used.

[0090] The operation of the image decoding device of the above embodiment in relation to image decoding will be described in more detail below. In this embodiment, the encoded bitstream is input on a frame-by-frame basis.

[0091] In Figure 2, the bitstream for one frame input from the input terminal 201 is supplied to the separation and decoding unit 202. The separation and decoding unit 202 separates coded data related to the decoding process and coefficients from the bitstream and decodes the coded data present in the header portion of the bitstream. The separation and decoding unit 202 then supplies the quantization matrix coded data contained in the header portion to the quantization matrix decoding unit 209 and supplies the coded data of the image data to the decoding unit 203. Specifically, the separation and decoding unit 202 first extracts the quantization matrix coded data from the sequence header of the bitstream shown in Figure 6(a) and outputs the extracted quantization matrix coded data to the quantization matrix decoding unit 209. In this embodiment, quantization matrix coded data corresponding to the quantization matrices shown in Figures 8(a) to 8(l) is extracted and output. Subsequently, coded data for the basic block units of the picture data is extracted and output to the decoding unit 203.

[0092] The quantization matrix decoding unit 209 first decodes the input quantization matrix coded data and, based on the syntax table shown in Figure 12, reconstructs the coded results of each quantization matrix generated on the encoding side. As shown in Figure 6(a), the coded result of each quantization matrix consists of a quantization matrix coding mode and data corresponding to each quantization matrix coding mode.

[0093] For example, if the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table in Figure 12, is 0, it means that the quantization matrix was coded in quantization matrix reference mode. In this case, the coded result of the quantization matrix consists of the quantization matrix coding mode (0) and the quantization matrix reference index. For example, when decoding a 4x4 quantization matrix for the intra-prediction Cb component, the quantization matrix decoding unit 209 first reconstructs the quantization matrix coding mode = 0, which represents the quantization matrix reference mode. Then, the quantization matrix decoding unit 209 reconstructs the quantization matrix reference index. If the quantization matrix reference index indicates that the referenced quantization matrix among the decoded quantization matrices was decoded immediately before, then the quantization matrix reference index = 0. If the quantization matrix reference index = 1, then the quantization matrix that was decoded one step earlier is referenced. Based on this information, the quantization matrix decoding unit 209 determines that the 4x4 quantization matrix for the intra-predicted Cb component is identical to the quantization matrix 806 in Figure 8(g) for the intra-predicted Y component, and then reconstructs the quantization matrix 801 in Figure 8(b).

[0094] Furthermore, when the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table of Figure 12, is 1, it means that the quantization matrix was coded in the inter-quantization matrix difference coding mode. In this case, the coded result of the quantization matrix consists of the quantization matrix coding mode (1), the quantization matrix reference index, and difference information. For example, when decoding a 4x4 quantization matrix for the intra-prediction Cr component, the quantization matrix decoding unit 209 first reconstructs the quantization matrix coding mode = 1, which means the inter-quantization matrix difference coding mode. Then, the quantization matrix decoding unit 209 reconstructs the quantization matrix reference index. If the quantization matrix reference index indicates that the referenced quantization matrix among the decoded quantization matrices was decoded immediately before, the quantization matrix reference index = 0. If the quantization matrix reference index = 1, the quantization matrix that was decoded one step earlier is referenced. Next, the quantization matrix decoding unit 209 decodes the difference information, i.e., the scaling_list_pred_delta in the syntax table of Figure 12, for the number of elements in the quantization matrix, and reconstructs the one-dimensional difference matrix shown in Figure 10(b). The quantization matrix decoding unit 209 uses the scanning method of Figure 9(b) to arrange the one-dimensional difference matrix of Figure 10(b) in two dimensions, obtaining the two-dimensional difference matrix shown in Figure 10(a). Then, the quantization matrix decoding unit 209 adds each difference value of the two-dimensional difference matrix of Figure 10(a) to each element of the quantization matrix 801 of Figure 8(b), which is the reference target, and reconstructs the quantization matrix 802 of Figure 8(c).

[0095] On the other hand, when the quantization matrix coding mode, i.e., scaling_list_pred_mode_idx in the syntax table of Figure 12, is 2, it means that the quantization matrix was coded in the intra-quantization matrix difference coding mode. In this case, the coded result of the quantization matrix consists of the quantization matrix coding mode (2) and difference information. For example, when decoding a 4x4 quantization matrix for intra-prediction Y component, the quantization matrix decoding unit 209 first reproduces the quantization matrix coding mode = 2, which means the intra-quantization matrix difference coding mode. Then, the quantization matrix decoding unit 209 decodes the difference information, i.e., scaling_list_delta_coef in the syntax table of Figure 12, for the number of elements in the quantization matrix, and reproduces the one-dimensional difference matrix shown in Figure 10(c). The quantization matrix decoding unit 209 adds each difference value in the one-dimensional difference matrix shown in Figure 10(b) to the immediately preceding element, arranges them in two dimensions using the scanning method shown in Figure 9(b), and reconstructs the quantization matrix 806 shown in Figure 8(g).

[0096] In this embodiment, the coding table in Figure 11(a) is used for decoding the differential information, but the coding table in Figure 11(b) may also be used. In short, it is fine to use the same one as the encoding side. The quantization matrix decoding unit 209 then reconstructs and holds the quantization matrices 800 to 811 in this manner. Here, it performs the opposite operation to the quantization matrix coding unit 113 on the encoding side.

[0097] The decoding unit 203 decodes the coded data supplied from the separation decoding unit 202, reconstructs the prediction information, and further reconstructs the residual coefficients. First, the decoding unit 203 reconstructs the prediction information and obtains the prediction mode used in the subblock. The decoding unit 203 outputs the reconstructed residual coefficients to the inverse quantization / inverse transform unit 204 and outputs the reconstructed prediction information to the image reconstruction unit 205.

[0098] The inverse quantization / inverse transformation unit 204 performs inverse quantization on the input residual coefficients using the quantization matrix reconstructed by the quantization matrix decoding unit 209 to generate orthogonal transformation coefficients, and then performs an inverse orthogonal transformation to reconstruct the prediction error. Similar to the inverse quantization / inverse transformation unit 106 on the encoding side, the inverse quantization / inverse transformation unit 204 performs inverse quantization using a quantization matrix corresponding to the size and color components of the subblock to be decoded. That is, when the intra-prediction mode is used without subblock partitioning, the quantization matrix shown in Figure 8(a) is used for the orthogonal transformation coefficients of the Y component, Figure 8(b) is used for the orthogonal transformation coefficients of the Cb component, and Figure 8(c) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the inter-prediction mode is used instead of subblock partitioning, the quantization matrix shown in Figure 8(d) is used for the orthogonal transformation coefficients of the Y component, Figure 8(e) is used for the orthogonal transformation coefficients of the Cb component, and Figure 8(f) is used for the orthogonal transformation coefficients of the Cr component. On the other hand, when the subblock partitioning shown in Figure 7(b) is performed and the intra-prediction mode is used, the quantization matrix shown in Figure 8(g) is used for the orthogonal transformation coefficients of the Y component, Figure 8(h) for the orthogonal transformation coefficients of the Cb component, and Figure 8(i) for the orthogonal transformation coefficients of the Cr component. Similarly, when the subblock partitioning shown in Figure 7(b) is performed, but the inter-prediction mode is used instead, the quantization matrix shown in Figure 8(j) is used for the orthogonal transformation coefficients of the Y component, Figure 8(k) for the orthogonal transformation coefficients of the Cb component, and Figure 8(l) for the orthogonal transformation coefficients of the Cr component are used.

[0099] The reconstructed orthogonal transformation coefficients are then subjected to an inverse orthogonal transformation, and the resulting reconstructed prediction error is output to the image reconstruction unit 205. However, the quantization matrix used is not limited to this; it can be the same as the quantization matrix used in the encoding-side transformation / quantization unit 105 and the inverse quantization / inverse transformation unit 106.

[0100] The image playback unit 205 appropriately references the frame memory 206 based on the prediction information input from the decoding unit 203 and plays back the predicted image. In this embodiment, the image playback unit 205 uses intra-prediction or inter-prediction, similar to the prediction unit 104 on the encoding side. The specific prediction process is the same as that of the prediction unit 104 on the encoding side, so a detailed explanation is omitted. The image playback unit 205 plays back image data from this predicted image and the prediction error input from the inverse quantization / inverse transform unit 204 and stores it in the frame memory 206. The stored image data is used for reference during prediction.

[0101] The in-loop filter unit 207, like the encoding-side in-loop filter unit 109, reads the replayed image from the frame memory 206 and performs in-loop filtering, such as deblocking filtering. The in-loop filter unit 207 then restores the filtered image back into the frame memory 206.

[0102] The playback image stored in the frame memory 206 is ultimately output to an external device (primarily a display device) via the output terminal 208.

[0103] Figure 4 is a flowchart showing the decoding process of the control unit 250 in the image decoding device according to the embodiment.

[0104] First, in S401, the control unit 250 controls the separation and decoding unit 202 to separate coded data related to the decoding process and coefficients from the bitstream and decode the coded data of the header portion. More specifically, the separation and decoding unit 202 supplies the coded data of the quantization matrix to the quantization matrix decoding unit 209 and the coded data of the image to the decoding unit 203.

[0105] In S402, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix code data reconstructed in S401 based on the syntax table in Figure 12, and reconstructs and retains the quantization matrices 801 to 811 shown in Figures 8(a) to (l). The specific operation of the quantization matrix decoding unit 209 here has already been explained and will be omitted here.

[0106] In S403, the control unit 250 controls the decoding unit 203 to decode the coded data separated in S401, reconstruct the prediction information, and reconstruct the residual coefficients.

[0107] In S404, the control unit 250 controls the inverse quantization / inverse transformation unit 204 to perform inverse quantization on the residual coefficients using the quantization matrix reconstructed in S402 to generate orthogonal transformation coefficients. The inverse quantization / inverse transformation unit 204 further performs an inverse orthogonal transformation to reconstruct the prediction error. In this embodiment, the quantization matrix used in the inverse quantization process is determined according to the color components and size of the subblock to be decoded. That is, the inverse quantization / inverse transformation unit 204 performs inverse quantization using the quantization matrices shown in Figures 8(a) to 8(l) according to the size, prediction mode, and color components of the subblock. However, the quantization matrix used is not limited to these and may be the same as the quantization matrix used on the encoding side.

[0108] In S405, the control unit 250 controls the image playback unit 205 to reproduce an image based on the prediction information generated in S403. Specifically, the image playback unit 205 reproduces a predicted image by referring to the frame memory 206 based on the prediction information. At this time, the image playback unit 205 uses intra-prediction or inter-prediction, similar to S305 on the encoding side. The image playback unit 205 then reproduces image data from the reproduced predicted image and the prediction error generated in S404, and stores the reproduced image data in the frame memory 206.

[0109] In S406, the control unit 250 determines whether or not the decoding of all basic blocks in the frame of interest has been completed. If it has been completed, the process proceeds to S407. If there are unsigned basic blocks, the process returns to S403 to select the next basic block for decoding.

[0110] In S407, the control unit 250 controls the in-loop filter unit 207 to perform in-loop filtering on the image data reproduced in S405, generate a filtered image, and terminate the process.

[0111] With the above configuration and operation, it is possible to decode the encoded bitstream generated by the image encoding device described earlier, that is, a bitstream encoded in an appropriate quantization matrix encoding mode, with the amount of code generated by the quantization matrix suppressed.

[0112] In this embodiment, only quantization matrices of the same size can be referenced in the quantization matrix reference mode and the inter-quantization matrix difference coding mode. However, it is also possible to configure the system to reference quantization matrices of different sizes using scaling. For example, when decoding the quantization matrix 806 in Figure 8(g), the quantization matrix 807 in Figure 8(h) can be referenced and scaled, the difference between the two can be decoded as difference information, and the quantization matrix can be reconstructed. This makes it possible to decode a bitstream with an even smaller amount of data in the quantization matrix initially encoded for each size.

[0113] Furthermore, in this embodiment, as shown in the syntax table in Figure 12, the system is configured to decode from the smallest quantization matrix first, but it is also possible to configure it to decode from the largest quantization matrix first. This makes it possible to decode a bitstream with an even smaller amount of data in the quantization matrix, especially when referencing between quantization matrices of different sizes is enabled as described above.

[0114] Furthermore, in this embodiment, the reference target of the quantization matrix is ​​limited to the decoded quantization matrix, but as shown in Figures 13(a) to (c), a flat quantization matrix that serves as a reference when a quantization matrix is ​​not used can also be used as the reference target. This makes it possible to decode a bitstream with an even smaller amount of data in the quantization matrix, especially when the quantization matrix to be decoded is close to a flat quantization matrix.

[0115] [Second Embodiment] In the first embodiment described above, the processing units of the image encoding device and image decoding device were described as being composed of hardware. However, the processing performed by each of the processing units shown in these figures may also be composed of computer programs.

[0116] Figure 5 is a block diagram showing an example of computer hardware configuration applicable to the image encoding device and decoding device according to the above embodiment.

[0117] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and ROM 503, and also executes the processes described above as being performed by the image processing device according to the above embodiment. In other words, the CPU 501 functions as the processing unit shown in Figures 1 and 2.

[0118] RAM 502 has an area for temporarily storing data acquired from external sources via external storage device 506 and interface 507. Furthermore, RAM 502 is also used as a work area when the CPU 501 performs various processes. RAM 502 can be allocated as frame memory, for example, or various other areas can be provided as appropriate.

[0119] ROM 503 stores the computer's configuration data and boot program. The operation unit 504 consists of a keyboard and mouse, allowing the computer's user to input various instructions to the CPU 501. The display unit 505 displays the processing results from the CPU 501. The display unit 505 is, for example, a liquid crystal display.

[0120] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores the OS (operating system) and computer programs (application programs) that enable the CPU 501 to implement the functions of each part shown in Figures 1 and 2. Furthermore, the external storage device 506 may also store the image data to be processed.

[0121] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate, according to the control of the CPU 501, and become subject to processing by the CPU 501. Other devices such as LANs, the Internet, projection devices, and display devices can be connected to the I / F 507, and this computer can acquire and transmit various information via this I / F 507. 508 is a bus that connects the above-mentioned parts.

[0122] In the above configuration, when power is supplied to the device, the CPU 501 executes the boot program stored in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and executes it. Then, under the control of the OS, the CPU 501 loads the application program related to encoding or decoding from the external storage device 506 into the RAM 502 and executes it. As a result, the CPU 501 functions as the respective processing units shown in Figure 1 or Figure 2, and the device functions as an image encoding device or an image decoding device.

[0123] Here, the encoding process of the quantization matrix in this second embodiment, that is, the process corresponding to S302 in Figure 3, will be explained with reference to the flowchart in Figure 14.

[0124] The order of the quantization matrices to be encoded is the same as in the first embodiment: first, the 2x2 pixel size quantization matrices 807→808→810→811 are used; then, the 4x4 pixel size quantization matrices 806→801→802→809→804→806 are used; and finally, the 8x8 pixel size quantization matrices 800→803 are used.

[0125] Furthermore, the variables sizeId and matrixId shown below have the same meaning as those shown in the first embodiment.

[0126] In other words, the variable `sizeId` indicates the size of the quantization matrix. When `sizeId` is "1", it indicates that the quantization matrix to be encoded is 2x2 pixels in size. When `sizeId` is "2", it indicates that the quantization matrix to be encoded is 4x4 pixels in size. When `sizeId` is "3", it indicates that the quantization matrix to be encoded is 8x8 pixels in size. The variable `matrixId` indicates the order of quantization matrices of the same size. Thus, the quantization matrix to be encoded is identified by `sizeId` and `matrixId`.

[0127] For example, when sizeId=2 and matrixId=0, it indicates that the first quantization matrix 806, which is 4x4 pixels in size, is the quantization matrix to be encoded.

[0128] First, in S1401, CPU501 sets the size of the quantization matrix to be encoded to 2x2 pixels by setting the variable sizeId to "1" as its initial value. Next, in S1402, CPU501 sets matrixId to "0" as its initial value. As a result, the quantization matrix 807 shown in Figure 8 is set as the quantization matrix of interest to be encoded when S1402 is executed for the first time.

[0129] In S1403, CPU 501 determines whether matrixId is "0". If matrixId is not "0", it means that there is an encoded quantization matrix of the same size as the quantization matrix of interest. Therefore, CPU 501 proceeds to S1404 and performs a search using quantization matrix reference mode. In other words, CPU 501 searches whether there is an encoded quantization matrix that is identical to the quantization matrix of interest.

[0130] In S1405, the CPU 501 determines whether it was able to find an encoded quantization matrix identical to the quantization matrix of interest. If it determines that it was found, the CPU 501 proceeds to S1406 and outputs "0" and a value that identifies the matching quantization matrix as encoded data for the quantization matrix of interest. Setting "0" at the beginning of the encoded data is equivalent to setting the scaling_list_pred_mode_idx parameter to "0". Also, in this embodiment, since the number of quantization matrices of size 4x4 pixels, "6", is greater than the number of quantization matrices of other sizes, it is sufficient to allocate 3 bits to the value that identifies the matching quantization matrix. On the other hand, if it determines that it was not found, the CPU 501 proceeds to S1407.

[0131] If CPU501 determines in S1403 that matrixId is "0", the process proceeds to S1409.

[0132] In S1407, CPU 501 executes the inter-quantization matrix difference coding mode for the quantization matrix of interest. Then, in S1408, CPU 501 obtains "1" and coded data using the inter-quantization matrix difference coding mode. Setting "1" at the beginning of the coded data is equivalent to setting the parameter of scaling_list_pred_mode_idx to "1". Alternatively, information indicating the order of the quantization matrix, which is the same as the quantization matrix of interest, may be output following the leading "1", and then the difference coded data may be output.

[0133] Next, in S1409, CPU 501 executes the intra-quantization matrix difference coding mode for the quantization matrix of interest. Then, in S1410, CPU 501 obtains "2" and the coded data using the inter-quantization matrix difference coding mode. Setting "2" at the beginning of the coded data is equivalent to setting the return value of scaling_list_pred_mode_idx to "2".

[0134] Then, in S1411, CPU 501 outputs the smaller of the two encoded data generated in S1408 and S1410 as the encoded data of the quantization matrix of interest. If the process branches from S1403 to S1409, the encoded data generated in the quantization matrix differential encoding mode of S1409 is output.

[0135] In S1412, CPU 501 increments the variable matrixId by "1". Then, in S1413, CPU 501 determines, based on the variables sizeId and matrixId, whether it has finished encoding all quantization matrices of the same size as the quantization matrix of interest. If the result of the determination in S1413 is "No", CPU 501 proceeds to S1403 and repeats the above process.

[0136] Furthermore, if the result of the S1413 determination is "Yes", the CPU 501 increments sizeId by "1". Then, in S1415, the CPU 501 determines, based on the variables sizeId and matrixId, whether or not it has finished encoding all sizes of quantization matrices. If the result of the S1415 determination is "No", it means that there are quantization matrices of unencoded sizes, so the CPU returns to processing S1402. If the result of the S1415 determination is "Yes", this process (S302 in Figure 3) is completed.

[0137] Note that if the process proceeds to S1406, it is guaranteed that the amount of data encoded using the quantization matrix reference mode will decrease, so the comparison with data encoded in other modes has been omitted, but of course, it is permissible to make such a comparison.

[0138] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Industrial applicability]

[0139] This invention is used in encoding and decoding devices for encoding and decoding still images and videos. In particular, it can be applied to encoding and decoding methods that use quantization matrices. [Explanation of Symbols]

[0140] 101...Input terminal, 102...Block division unit, 103...Quantization matrix holding unit, 104...Prediction unit, 105...Conversion / quantization unit, 106...Inverse quantization / inverse conversion unit, 107...Image playback unit, 108...Frame memory, 109...In-loop filter unit, 110...Encoding unit, 111...Integrated encoding unit, 112...Output terminal, 113...Quantization matrix encoding unit, 150...Control unit

Claims

1. A decoding means for decoding the first quantization matrix, The system comprises an inverse quantization means that inversely quantizes the residual coefficients using the first quantization matrix, The decoding means decodes first information representing the decoding mode of the first quantization matrix from the scanning_list_data syntax structure, If the decoded first information indicates that the first mode is used, the decoding means decodes second information for identifying the second quantization matrix of reference from the scanning_list_data syntax structure, and derives the first quantization matrix by determining that the second quantization matrix identified based on the second information is the same as the first quantization matrix. If the decoded first information indicates that the second mode is used, the decoding means decodes the second information for identifying the second quantization matrix from the scanning_list_data syntax structure, (a) The first element of a first one-dimensional matrix consisting of multiple elements, which is the difference between the element in the first row and first column of the first quantization matrix and the element in the first row and first column of the second quantization matrix identified based on the second information, is associated with the first element in the first row and first column of a two-dimensional matrix as the first element in a predetermined scanning order, and the element in the first row and first column of the first quantization matrix is ​​derived using the first element associated with the first row and first column and the element in the first row and first column of the second quantization matrix, (b) The second element of the first one-dimensional matrix, which is the difference between the element in the second row and first column of the first quantization matrix and the element in the second row and first column of the second quantization matrix, is associated with the second element in the second row and first column of the two-dimensional matrix as the second element in the predetermined scanning order, and the element in the second row and first column of the first quantization matrix is ​​derived using the second element associated with the second row and first column and the element in the second row and first column of the second quantization matrix. (c) The third element of the first one-dimensional matrix, which is the difference between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix, is associated with the first row and second column of the two-dimensional matrix as the third element in the predetermined scanning order, and the first quantization matrix is ​​decoded by deriving the element in the first row and second column of the first quantization matrix using the third element associated with the first row and second column and the element in the first row and second column of the second quantization matrix. If the decoded first information indicates that the third mode is used, the decoding means decodes a plurality of difference values ​​from the scanning_list_data syntax structure, including the difference between the element in the first row and first column of the first quantization matrix and a predetermined value, and the difference between two consecutive elements in the predetermined scanning order of the first quantization matrix. (a) Using the first element of the second one-dimensional matrix obtained from the plurality of difference values, derive the element that corresponds to the first of the predetermined scanning order, which is the first row and first column of the first quantization matrix, (b) Using the second element of the second one-dimensional matrix, derive the element that corresponds to the second in the predetermined scan order, which is the second row and first column of the first quantization matrix, (c) Using the third element of the second one-dimensional matrix, derive the element that corresponds to the third in the predetermined scanning order, which is the first row and second column of the first quantization matrix. (d) The first quantization matrix is ​​decoded by using the fourth element of the second one-dimensional matrix to derive the element that is in the third row and first column of the first quantization matrix, which corresponds to the fourth element in the predetermined scanning order. The decoding means can use a quantization matrix in which all elements are set to 16 as the second quantization matrix. The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used. The size of the second quantization matrix is ​​limited to one of the following sizes: 2x2, 4x4, or 8x8. The 2x2 size second quantization matrix is ​​applicable to chroma blocks, but not to luma blocks. When decoding the 2x2 size second quantization matrix, the 4x4 size second quantization matrix, and the 8x8 size second quantization matrix, the 2x2 size second quantization matrix is ​​decoded first. If the size of the first quantization matrix is ​​4x4, then the second quantization matrix is ​​4x4. An image decoding device characterized by the following features.

2. A quantization means that quantizes the transformation coefficients of blocks in an image using a first quantization matrix, The system comprises encoding means for encoding the first quantization matrix, When the first mode is used to encode the first quantization matrix, the encoding means encodes information indicating that the first quantization matrix is ​​identical to the second quantization matrix of reference and information for identifying the second quantization matrix into the scanning_list_data syntax structure. When the second mode is used to encode the first quantization matrix, the encoding means encodes information for identifying the second quantization matrix in the scanning_list_data syntax structure, (a) The first element in a predetermined scanning order, which is the element of the difference between the element in the first row and first column of the first quantization matrix and the element in the first row and first column of the second quantization matrix, (b) The second element in the predetermined scanning order, which is the element of the difference between the element in the second row and first column of the first quantization matrix and the element in the second row and first column of the second quantization matrix, (c) Encode information used to derive a first one-dimensional matrix consisting of a plurality of elements, the third element in the predetermined scanning order which is a third element that is the difference between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix, When a third mode is used to encode the first quantization matrix, the encoding means, (a) The difference between the element in the first row and first column of the first quantization matrix that corresponds to the first element in the predetermined scanning order and a predetermined value is encoded in the scanning_list_data syntax structure, (b) The difference between the second element in the second row and first column of the first quantization matrix and the first element in the first row and first column of the first quantization matrix, which corresponds to the second element in the predetermined scanning order, is encoded in the scanning_list_data syntax structure. (c) The difference between the element in the first row and second column of the first quantization matrix that corresponds to the third element in the predetermined scanning order and the element in the second row and first column of the first quantization matrix is ​​encoded in the scanning_list_data syntax structure. (d) The difference between the element in the third row and first column of the first quantization matrix that corresponds to the fourth element in the predetermined scanning order and the element in the first row and second column of the first quantization matrix is ​​encoded in the scanning_list_data syntax structure. The difference between two consecutive elements in the predetermined scanning order of the first quantization matrix is ​​encoded, The encoding means can use a quantization matrix in which all elements are set to 16 as the second quantization matrix. The code size of the information indicating that the first mode is used is smaller than the code size of the information indicating that the second mode is used. The size of the second quantization matrix is ​​limited to one of the following sizes: 2x2, 4x4, or 8x8. The 2x2 size second quantization matrix is ​​applicable to chroma blocks, but not to luma blocks. When encoding the 2x2 size second quantization matrix, the 4x4 size second quantization matrix, and the 8x8 size second quantization matrix, the 2x2 size second quantization matrix is ​​encoded first. If the size of the first quantization matrix is ​​4x4, then the second quantization matrix is ​​4x4. An image coding device characterized by the following:

3. A decoding step for decoding the first quantization matrix, The process includes an inverse quantization step of inversely quantizing the residual coefficients using the first quantization matrix, The decoding step decodes first information representing the decoding mode of the first quantization matrix from the scanning_list_data syntax structure, If the decoded first information indicates that the first mode is used, in the decoding step, second information for identifying the second quantization matrix of reference from the scanning_list_data syntax structure is decoded, and the first quantization matrix is ​​derived by determining that the second quantization matrix identified based on the second information is the same as the first quantization matrix. If the decoded first information indicates that the second mode is used, in the decoding step, the second information for identifying the second quantization matrix from the scanning_list_data syntax structure is decoded, (a) The first element of a first one-dimensional matrix consisting of multiple elements, which is the difference between the element in the first row and first column of the first quantization matrix and the element in the first row and first column of the second quantization matrix identified based on the second information, is associated with the first element in the first row and first column of a two-dimensional matrix as the first element in a predetermined scanning order, and the element in the first row and first column of the first quantization matrix is ​​derived using the first element associated with the first row and first column and the element in the first row and first column of the second quantization matrix, (b) The second element of the first one-dimensional matrix, which is the difference between the element in the second row and first column of the first quantization matrix and the element in the second row and first column of the second quantization matrix, is associated with the second element in the second row and first column of the two-dimensional matrix as the second element in the predetermined scanning order, and the element in the second row and first column of the first quantization matrix is ​​derived using the second element associated with the second row and first column and the element in the second row and first column of the second quantization matrix. (c) The third element of the first one-dimensional matrix, which is the difference between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix, is associated with the first row and second column of the two-dimensional matrix as the third element in the predetermined scanning order, and the first quantization matrix is ​​decoded by deriving the element in the first row and second column of the first quantization matrix using the third element associated with the first row and second column and the element in the first row and second column of the second quantization matrix. If the decoded first information indicates that the third mode is used, in the decoding step, a plurality of difference values, including the difference between the element in the first row and first column of the first quantization matrix and a predetermined value, and the difference between two consecutive elements in the predetermined scanning order of the first quantization matrix, are decoded from the scanning_list_data syntax structure. (a) Using the first element of the second one-dimensional matrix obtained from the plurality of difference values, derive the element that corresponds to the first of the predetermined scanning order, which is the first row and first column of the first quantization matrix, (b) Using the second element of the second one-dimensional matrix, derive the element that corresponds to the second in the predetermined scan order, which is the second row and first column of the first quantization matrix, (c) Using the third element of the second one-dimensional matrix, derive the element that corresponds to the third in the predetermined scanning order, which is the first row and second column of the first quantization matrix. (d) The first quantization matrix is ​​decoded by using the fourth element of the second one-dimensional matrix to derive the element that is in the third row and first column of the first quantization matrix, which corresponds to the fourth element in the predetermined scanning order. In the decoding step, a quantization matrix in which all elements are set to 16 can be used as the second quantization matrix. The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used. The size of the second quantization matrix is ​​limited to one of the following sizes: 2x2, 4x4, or 8x8. The 2x2 size second quantization matrix is ​​applicable to chroma blocks, but not to luma blocks. When decoding the 2x2 size second quantization matrix, the 4x4 size second quantization matrix, and the 8x8 size second quantization matrix, the 2x2 size second quantization matrix is ​​decoded first. If the size of the first quantization matrix is ​​4x4, then the second quantization matrix is ​​4x4. An image decoding method characterized by the following:

4. A quantization step in which the transformation coefficients of blocks in the image are quantized using a first quantization matrix, The process includes an encoding step for encoding the first quantization matrix, When the first mode is used to encode the first quantization matrix, in the encoding step, information indicating that the first quantization matrix is ​​identical to the second quantization matrix of reference and information for identifying the second quantization matrix are encoded in the scanning_list_data syntax structure. When the second mode is used to encode the first quantization matrix, in the encoding step, information for specifying the second quantization matrix is ​​encoded in the scanning_list_data syntax structure, (a) The first element in a predetermined scanning order, which is the element of the difference between the element in the first row and first column of the first quantization matrix and the element in the first row and first column of the second quantization matrix, (b) The second element in the predetermined scanning order, which is the element of the difference between the element in the second row and first column of the first quantization matrix and the element in the second row and first column of the second quantization matrix, (c) Encode information used to derive a first one-dimensional matrix consisting of a plurality of elements, the third element in the predetermined scanning order which is a third element that is the difference between the element in the first row and second column of the first quantization matrix and the element in the first row and second column of the second quantization matrix, When a third mode is used to encode the first quantization matrix, in the encoding step, (a) The difference between the element in the first row and first column of the first quantization matrix that corresponds to the first element in the predetermined scanning order and a predetermined value is encoded in the scanning_list_data syntax structure, (b) The difference between the second element in the second row and first column of the first quantization matrix and the first element in the first row and first column of the first quantization matrix, which corresponds to the second element in the predetermined scanning order, is encoded in the scanning_list_data syntax structure. (c) The difference between the element in the first row and second column of the first quantization matrix that corresponds to the third element in the predetermined scanning order and the element in the second row and first column of the first quantization matrix is ​​encoded in the scanning_list_data syntax structure. (d) The difference between the element in the third row and first column of the first quantization matrix that corresponds to the fourth element in the predetermined scanning order and the element in the first row and second column of the first quantization matrix is ​​encoded in the scanning_list_data syntax structure. The difference between two consecutive elements in the predetermined scanning order of the first quantization matrix is ​​encoded, In the encoding process, a quantization matrix in which all elements are set to 16 can be used as the second quantization matrix. The code size of the information indicating that the first mode is used is smaller than the code size of the information indicating that the second mode is used. The size of the second quantization matrix is ​​limited to one of the following sizes: 2x2, 4x4, or 8x8. The 2x2 size second quantization matrix is ​​applicable to chroma blocks, but not to luma blocks. When encoding the 2x2 size second quantization matrix, the 4x4 size second quantization matrix, and the 8x8 size second quantization matrix, the 2x2 size second quantization matrix is ​​encoded first. If the size of the first quantization matrix is ​​4x4, then the second quantization matrix is ​​4x4. An image encoding method characterized by the following.

5. A program that, when read and executed by a computer, causes the computer to perform each step of the method according to claim 3.

6. A program that, when read and executed by a computer, causes the computer to perform each step of the method according to claim 4.

Images