Image decoding apparatus, image encoding apparatus, control method thereof, and program

Abstract

To reduce the code amount of a quantization matrix to improve the efficiency of coding image data.SOLUTION: An image encoder has a conversion and quantization unit that quantizes a prediction error of a block by using a quantization matrix, and an encoding unit that encodes the quantization matrix. The encoding unit has a first mode for encoding information indicating that the quantization matrix to be encoded is the same as an encoded quantization matrix and not encoding elements of the quantization matrix to be encoded, a second mode for encoding information indicating encoding the difference between the elements of the quantization matrix to be encoded and elements of the encoded quantization matrix and encoding the difference between the elements of the quantization matrix to be encoded and the elements of the encoded quantization matrix, and a third mode for encoding information indicating encoding the difference between the elements in the quantization matrix to be encoded and encoding the difference between the elements in the quantization matrix to be encoded.SELECTED DRAWING: None

Description

Technical Field

[0001] The present invention relates to image encoding processing and decoding processing.

Background Art

[0002] As an encoding method for compression recording of moving images, the HEVC (High Efficiency Video Coding) encoding method (hereinafter simply referred to as HEVC) is known. In HEVC, in order to improve the encoding efficiency, a basic block having a size larger than that of a conventional macroblock (16×16 pixels) is adopted. This basic block having a large size is called a CTU (Coding Tree Unit), and its size is up to 64×64 pixels at maximum. A CTU is further divided into sub-blocks that are units for performing prediction and transformation.

[0003] Also, in HEVC, a process of weighting coefficients (hereinafter referred to as orthogonal transform coefficients) after orthogonal transform, called a quantization matrix, is used according to frequency components. By further reducing data of high-frequency components that are less likely to be noticeable in human vision degradation, it is possible to increase the compression efficiency while maintaining the image quality. Patent Document 1 discloses a technique for encoding such a quantization matrix.

[0004] In recent years, activities for further international standardization of a more efficient encoding method as a successor to HEVC have been started. JVET (Joint Video Experts Team) was established between ISO / IEC and ITU-T, and standardization is being promoted as the VVC (Versatile Video Coding) encoding method (hereinafter referred to as VVC). In order to improve the encoding efficiency, in VVC, the size of the basic block is up to 128×128 pixels at maximum, and in addition to the conventional square sub-block division, rectangular sub-block division is also being considered.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2013-38758 [Summary of the Invention] [Problems to be Solved by the Invention]

[0006] In VVC as well, the introduction of a quantization matrix is being considered in the same way as in HEVC. Furthermore, in VVC, a greater variety of sub-block divisions, including rectangular shapes, are being considered compared to HEVC. Since the distribution of orthogonal transform coefficients corresponding to each sub-block division differs depending on the size and shape of the orthogonal transform, it is desirable to define an optimal quantization matrix according to the size and shape of the sub-blocks. However, if an individual quantization matrix is defined for all sub-block shapes, the amount of code for the quantization matrix will increase unnecessarily.

[0007] The present invention has been made in view of such problems, and aims to provide a technique for suppressing the amount of code for the quantization matrix. [Means for Solving the Problems]

[0008] To solve this problem, for example, the image decoding apparatus of the present invention has the following configuration. That is, decoding means for decoding a first quantization matrix, and inverse quantization means for inverse quantizing residual coefficients using the first quantization matrix, wherein the decoding means from the scaling_list_data syntax structure decodes information representing the mode of decoding of the first quantization matrix, First and when the decoded information indicates that the first mode is used, the decoding means First derives the first quantization matrix by determining that the second quantization matrix is the same as the first quantization matrix, decodes second information for identifying a second quantization matrix to be referred to from the scaling_list_data syntax structure, and based on the second information, the and when the decoded information indicates that the second mode is used, the decoding means First decodes the second information for identifying the second quantization matrix from the scaling_list_data syntax structure, (a) A first element in a first one-dimensional matrix composed of a plurality of elements, which is an element of a difference value between an element at the first row and first column of the first quantization matrix and an element at the first row and first column of the second quantization matrix identified based on the second information, is associated with the first row and first column of a two-dimensional matrix as the first element in a predetermined scanning order, and using the first element associated with the first row and first column and the element at the first row and first column of the second quantization matrix, derives the element at the first row and first column of the first quantization matrix, (b) A second element in the first one-dimensional matrix, which is an element of a difference value between an element at the second row and first column of the first quantization matrix and an element at the second row and first column of the second quantization matrix, is associated with the second row and first column of the two-dimensional matrix as the second element in the predetermined scanning order, and using the second element associated with the second row and first column and the element at the second row and first column of the second quantization matrix, derives the element at the second row and first column of the first quantization matrix, (c) A third element in the first one-dimensional matrix, which is an element of a difference value between an element at the first row and second column of the first quantization matrix and an element at the first row and second column of the second quantization matrix, is associated with the first row and second column of the two-dimensional matrix as the third element in the predetermined scanning order, and using the third element associated with the first row and second column and the element at the first row and second column of the second quantization matrix, derives the element at the first row and second column of the first quantization matrix in this way, Decode the first quantization matrix, When the decoded First information indicates that the third mode is used, the decoding means decodes a plurality of difference values including a difference value between an element at the first row and first column of the first quantization matrix and a predetermined value and a difference value between two consecutive elements in the predetermined scanning order of the first quantization matrix from the scaling_list_data syntax structure, (a) Using the first element in the second one-dimensional matrix obtained from the plurality of difference values, derive the element at the first row and first column of the first quantization matrix corresponding to the first in the predetermined scanning order. (b) Using the second element in the second one-dimensional matrix, derive the element at the second row and first column of the first quantization matrix corresponding to the second in the predetermined scanning order. (c) Using the third element in the second one-dimensional matrix, derive the element at the first row and second column of the first quantization matrix corresponding to the third in the predetermined scanning order. (d) Using the fourth element in the second one-dimensional matrix, derive the element at the third row and first column of the first quantization matrix corresponding to the fourth in the predetermined scanning order. Decode the first quantization matrix, The decoding means can use a quantization matrix in which all elements are set to 16 as the second quantization matrix, The amount of code of the information indicating that the first mode is used First is smaller than the amount of code of the information indicating that the second mode is used First information. It is characterized by this.

Advantages of the Invention

[0009] According to the present invention, it becomes possible to suppress the amount of code of the quantization matrix more than before and improve the coding efficiency of image data.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and redundant explanations are omitted.

[0012] [First Embodiment] Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and redundant explanations are omitted.

[0013] FIG. 1 is a block diagram of an image encoding apparatus according to a first embodiment. The image encoding apparatus includes a control unit 150 that controls the entire apparatus. The control unit 150 includes a CPU, a ROM that stores programs executed by the CPU, and a RAM used as a work area for the CPU. The image encoding apparatus also includes an input terminal 101, a block division unit 102, a quantization matrix holding unit 103, a prediction unit 104, a transform / quantization unit 105, an inverse quantization / inverse transform unit 106, an image reproduction unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0014] The input terminal 101 inputs the image data to be encoded generated by an image data source in units of frames. The image data source may be of any type, such as an imaging device, a file server storing the image data to be encoded, or a storage medium. The output terminal 112 outputs the encoded data to an output destination device, which may also be of any type, such as a storage medium or a file server.

[0015] The block division unit 102 divides the image of the input frame into a plurality of basic blocks and sequentially outputs one of them as a basic block to the subsequent prediction unit 104.

[0016] The quantization matrix holding unit 103 generates a plurality of quantization matrices prior to encoding and holds them in an internal memory (not shown). The method for generating the quantization matrices is not particularly limited. The user may input the quantization matrices, or they may be calculated from the characteristics of the input image, or those specified in advance as initial values may be used. The quantization matrix holding unit 103 in this embodiment generates and holds two-dimensional quantization matrices 800 to 811 corresponding to orthogonal transforms of 8×8 pixel size, 4×4 pixel size, or 2×2 pixel size shown in FIGS. 8(a) to (l). Here, quantization matrices 800, 803, 806, 809 are for luminance components, and quantization matrices 801, 802, 804, 805, 807, 808, 810, 811 are for two color difference components.

[0017] The prediction unit 104 determines sub-block division for the image data in basic block units, performs intra prediction which is in-frame prediction in sub-block units, inter prediction which is inter-frame prediction, etc., and generates predicted image data. Further, the prediction unit 104 calculates and outputs the prediction error in pixel units between the sub-blocks in the input image data and the corresponding predicted image data. Also, the prediction unit 104 outputs information necessary for prediction, such as sub-block division, prediction mode, motion vector, etc., together with the prediction error. Hereinafter, this information necessary for prediction is referred to as prediction information.

[0018] The transform and quantization unit 105 orthogonally transforms the prediction error of the sub-blocks input from the prediction unit 104 in sub-block units to obtain orthogonal transform coefficients. Further, the transform and quantization unit 105 performs quantization of the orthogonal transform coefficients using the quantization matrix stored in the quantization matrix holding unit 103 to obtain residual coefficients (quantized orthogonal transform coefficients).

[0019] The inverse quantization and inverse transform unit 106 inputs the residual coefficients from the transform and quantization unit 105, performs inverse quantization using the corresponding quantization matrix stored in the quantization matrix holding unit 103, and reproduces the orthogonal transform coefficients. The inverse quantization and inverse transform unit 106 further performs inverse orthogonal transformation on the orthogonal transform coefficients to reproduce the prediction error.

[0020] The image reproduction unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image reproduction unit 107 generates reproduced image data by adding the prediction error input from the inverse quantization and inverse transform unit 106 to this predicted image data, and stores it in the frame memory 108.

[0021] The in-loop filter 109 performs in-loop filter processing such as deblocking filter and sample adaptive offset on the reproduced image stored in the frame memory 108, and stores the image data after the filter processing in the frame memory 108 again.

[0022] The symbolization unit 110 encodes the residual coefficients output from the conversion / quantization unit 105 and the prediction information output from the prediction unit 104 to generate symbol data, and outputs the symbol data to the integrated encoding unit 111.

[0023] The quantization matrix encoding unit 113 encodes the quantization matrix (see FIGS. 8(a) to (l)) held in the quantization matrix holding unit 103 to generate quantization matrix symbol data, and outputs the quantization matrix symbol data to the integrated encoding unit 111.

[0024] The integrated encoding unit 111 generates header symbol data including the quantization matrix symbol data from the quantization matrix encoding unit 113. Then, the integrated encoding unit 111 appends the symbol data output from the encoding unit 110 to the header symbol data to form a bit stream. Then, the integrated encoding unit 111 outputs the formed bit stream via the output terminal 112. Here, the image encoding operation in the image encoding apparatus will be described in more detail below. In the present embodiment, the input terminal 101 inputs moving image data in the 4:2:0 color format in units of frames at a predetermined frame rate (for example, 30 frames / second), but it may also be configured to input still image data for one frame. Further, for the sake of explanation in the present embodiment, in the block division unit 101, it will be described as dividing the image data input from the input terminal 101 into basic blocks of 8×8 pixels. That is, the basic block of 8×8 pixels will include pixels of the luminance (Y) component of 8×8 pixels and the color difference (Cb and Cr) components of 4×4 pixels. Note that this is for ease of understanding and is not limited to the above numerical values (sizes).

[0025] Prior to encoding the image, generation and encoding of the quantization matrix are performed.

[0026] The quantization matrix holding unit 103 first generates and holds a quantization matrix. Specifically, it generates a quantization matrix according to the size of the sub-block to be encoded and the type of prediction method. In this embodiment, it generates a quantization matrix corresponding to the basic block of 8×8 pixels that is not divided into the sub-blocks shown in FIG. 7(a), and a quantization matrix corresponding to the sub-block of 4×4 pixels obtained by quadtree splitting of the basic block shown in FIG. 7(b). That is, the quantization matrix holding unit 103 generates quantization matrices of 8×8 pixel size and 4×4 pixel size for the luminance (Y) component, and quantization matrices of 4×4 pixel size and 2×2 pixel size for the color difference (Cb and Cr) components. However, the generated quantization matrix is not limited to this, and quantization matrices corresponding to the shape of the sub-blocks, such as 4×8 and 8×4, may be generated. The method for determining each element constituting the quantization matrix is not particularly limited. For example, a predetermined initial value may be used, or it may be set individually. Also, it may be generated according to the characteristics of the image.

[0027] The quantization matrix holding unit 103 holds the plurality of types of quantization matrices generated in this way in an internal memory (not shown). FIG. 8(a) shows a quantization matrix 800 of 8×8 pixel size for the Y component using intra prediction. Also, FIG. 8(b) shows a quantization matrix 801 of 4×4 pixel size for the Cb component using intra prediction, and FIG. 8(c) shows a quantization matrix 802 of 4×4 pixel size for the Cr component using intra prediction.

[0028] Similarly, FIG. 8(d) shows a quantization matrix 803 of 8×8 pixel size for the Y component using inter prediction. Also, FIG. 8(e) shows a quantization matrix 804 of 4×4 pixel size for the Cb component using inter prediction, and FIG. 8(f) shows a quantization matrix 805 of 4×4 pixel size for the Cr component using inter prediction.

[0029] Furthermore, FIG. 8(g) shows a quantization matrix 806 of 4×4 pixel size for the Y component using intra prediction. Also, FIG. 8(h) shows a quantization matrix 807 of 2×2 pixel size for the Cb component using intra prediction, and FIG. 8(i) shows a quantization matrix 808 of 2×2 pixel size for the Cr component using intra prediction.

[0030] Similarly, FIG. 8(j) shows a quantization matrix 809 of 4×4 pixel size for the Y component using inter prediction. Also, FIG. 8(k) shows a quantization matrix 810 of 2×2 pixel size for the Cb component using inter prediction, and FIG. 8(l) shows a quantization matrix 811 of 2×2 pixel size for the Cr component using inter prediction.

[0031] For simplicity of explanation, it is assumed that the configuration is for 64 pixels of 8×8, 16 pixels of 4×4, and 4 pixels of 2×2, and each square within the thick frame represents each element constituting the quantization matrix. In this embodiment, it is assumed that the 12 types of quantization matrices shown in FIGS. 8(a) to (l) are held in a two-dimensional shape, but of course, each element within the quantization matrix is not limited to this. Also, depending on the size of the sub-block, it is possible to hold a plurality of quantization matrices for the same color component and the same prediction mode. Generally, since the quantization matrix realizes quantization processing according to human visual characteristics, the elements in the low-frequency part corresponding to the upper left part of the quantization matrix are small, and the elements in the high-frequency part corresponding to the lower right part are large, as shown in FIGS. 8(a) to (l).

[0032] The quantization matrix encoding unit 113 sequentially reads out the two-dimensional quantization matrices held in the quantization matrix holding unit 106, and determines the encoding mode of the quantization matrix used when encoding each quantization matrix. In the present embodiment, three types of quantization matrix encoding modes, namely, the "quantization matrix reference mode", the "inter-quantization matrix differential encoding mode", and the "intra-quantization matrix differential encoding mode", are used. The quantization matrix encoding unit 113 determines the quantization matrix encoding mode that minimizes the data amount of the encoding result for each quantization matrix.

[0033] Here, the three types of quantization matrix encoding modes used by the quantization matrix encoding unit 113 of the present embodiment will be described in more detail.

[0034] The "quantization matrix reference mode" is a mode used when there is a quantized matrix in the encoded quantization matrices that matches the quantization matrix to be encoded. When this mode is used, instead of encoding each element of the quantization matrix to be encoded, the quantization matrix encoding unit 113 encodes the quantization matrix reference index, which is an identifier indicating the matching encoded quantization matrix to be referenced.

[0035] Next, the "inter-quantization matrix differential encoding mode" is a mode in which a quantization matrix close to the quantization matrix to be encoded is selected from the encoded quantization matrices, the difference value of each element is calculated, and the calculated difference value is encoded. When this mode is selected, the quantization matrix encoding unit 113 encodes the quantization matrix reference index, which is an identifier indicating the quantization matrix to be referenced, and then encodes the difference information, which is a set of the difference values of each element.

[0036] Then, the "differential encoding mode in the quantization matrix" is used when there is no encoded quantization matrix or when there is no quantization matrix similar to the quantization matrix to be encoded. The quantization matrix encoding unit scans each element of the quantization matrix to be encoded, calculates the difference value between the elements, and encodes the difference information, which is a set of difference values.

[0037] FIG. 12 is a syntax table used for encoding the quantization matrix in this embodiment. Using this syntax table, the encoding process of each quantization matrix shown in FIGS. 8(a) to (l) in this embodiment will be specifically described. First, the structure of the syntax table in FIG. 12 will be described. There is a for loop with a parameter sizeId on the outermost side of the syntax table. sizeId indicates the size of the quantization matrix, and quantization matrices of the same size will be encoded continuously. sizeId = 1 indicates a quantization matrix with a 2×2 pixel size. Similarly, sizeId = 2 indicates a quantization matrix with a 4×4 pixel size, and sizeId = 3 indicates a quantization matrix with an 8×8 pixel size.

[0038] The initial value of sizeId in the for loop is "1", which increases by "1" each time the loop iterates, and the loop repeats as long as sizeId is less than 4. Therefore, in this embodiment, after the quantization matrix group of 2×2 pixel size is encoded, the quantization matrix group of 4×4 pixel size is encoded, and then the quantization matrix group of 8×8 pixel size and so on are encoded in this order. In VVC, an orthogonal transform with a size ranging from a minimum of 2×2 pixel size to 64×64 pixel size is used. It is also possible to set up a for loop from the 2×2 pixel size with sizeId = 1 to the 64×64 pixel size with sizeId = 6 and encode the corresponding quantization matrices respectively. However, in this embodiment, since the maximum orthogonal transform size, that is, the size of the sub-block is 8×8, the upper limit value of the for loop is set to sizeId = 3 corresponding to the 8×8 pixel size. By setting the upper limit value of the for loop based on the maximum value of the actual orthogonal transform size used in this way, unnecessary encoding of quantization matrices can be omitted, and the generation of redundant codes can be prevented.

[0039] Inside the for loop with sizeId, there is a for loop with a parameter matrixId. This matrixId is a parameter based on the prediction mode and color component. When this value is 0, it means it is for intra prediction and Y component, when it is 1, it is for intra prediction and Cb component, and when it is 2, it is for intra prediction and Cr component. Similarly, when matrixId is 3, it means it is for inter prediction and Y component, when it is 4, it is for inter prediction and Cb component, and when it is 5, it is for inter prediction and Cr component. That is, within the quantization matrix group of the same size, first the quantization matrix group corresponding to intra prediction is encoded, and then the quantization matrix group corresponding to inter prediction is encoded. Furthermore, within the quantization matrix group corresponding to the same prediction mode, they are encoded in the order of Y component, Cb component, and Cr component.

[0040] Based on the encoding order of the quantization matrix above, the encoding order of each quantization matrix in FIGS. 8(a) to (l) is as follows.

[0041] First, FIGS. 8(h), (i), (k), and (l) with a 2×2 pixel size are encoded. Next, FIGS. 8(g), (b), (c), (j), (e), and (f) with a 4×4 pixel size are encoded. And finally, FIGS. 8(a) and (d) with an 8×8 pixel size are encoded.

[0042] The above encoding order is, in short, to encode in the order of priority of (i) the size of the quantization matrix, (ii) the type for intra / inter, and (iii) the type of component (luminance, color difference). The reason for setting the size of the quantization matrix as the first priority is for encoding considering the correlation between quantization matrices.

[0043] Next, the encoding procedure for each quantization matrix will be described. First, the quantization matrix encoding unit 113 encodes the determined quantization matrix encoding mode. The scaling_list_pred_mode_idx in the syntax table of FIG. 12 corresponds to the quantization matrix encoding mode. In this embodiment, when this value is 0, the "quantization matrix reference mode" is used, when it is 1, the "differential encoding mode between quantization matrices" is used, and when it is 2, the "differential encoding mode within the quantization matrix" is used. Since the quantization matrix encoding mode is Golomb-encoded, the smaller this value, the smaller the amount of code related to scaling_list_pred_mode_idx. That is, in this embodiment, the amount of code related to the matrix reference mode is the smallest, which has the merit of minimizing the generated amount of code when there are many identical quantization matrices.

[0044] When using the quantization matrix reference mode, the quantization matrix encoding unit 113 subsequently encodes a quantization matrix reference index. This corresponds to scaling_list_pred_matrix_id_delta in the syntax table of FIG. 12. For example, the quantization matrix 801 in FIG. 8(b) is the same as the quantization matrix 806 in FIG. 8(g) encoded immediately before. Therefore, when encoding the quantization matrix 801 in FIG. 8(b), the quantization matrix encoding unit 113 selects the quantization matrix reference mode and encodes 0 indicating that it is the quantization matrix encoded immediately before as the quantization matrix reference index. By doing so, a quantization matrix identical to the quantization matrix 806 in FIG. 8(g) can be encoded. Note that when using the quantization matrix one further before, the quantization matrix reference index becomes "1".

[0045] On the other hand, when using the quantization matrix differential coding mode, similar to the case of using the quantization matrix reference mode, the quantization matrix encoding unit 113 first encodes the quantization matrix reference index. Subsequently, the quantization matrix encoding unit 113 encodes the differential information, which is the difference between each element of the encoded reference quantization matrix determined by the quantization matrix reference index and each element of the quantization matrix to be encoded. This corresponds to scaling_list_pred_delta in the syntax table of FIG. 12. For example, each element of the quantization matrix 802 in FIG. 8(c) is similar to each element of the quantization matrix 801 in FIG. 8(b) encoded immediately before (in this example, the absolute value of the difference between each element is below a predetermined value (below "1" in the illustration)). Therefore, when encoding the quantization matrix 802 in FIG. 8(c), the quantization matrix encoding unit 113 selects the quantization matrix differential coding mode and first encodes 0 indicating that it is the quantization matrix encoded immediately before as the quantization matrix reference index. Note that when encoding the difference from each element of the quantization matrix one more before, the quantization matrix reference index becomes "1". Subsequently, the quantization matrix encoding unit 113 calculates the difference values between each element of the reference quantization matrix in FIG. 8(b) and each element of the quantization matrix to be encoded in FIG. 8(c), respectively, and calculates the two-dimensional difference value matrix shown in FIG. 10(a). Then, each difference value in the two-dimensional difference value matrix is scanned and arranged in a one-dimensional matrix. FIGS. 9(a) to 9(c) show the scanning methods used when arranging the two-dimensional matrix in one dimension in this embodiment. FIG. 9(a) corresponds to an 8×8, FIG. 9(b) corresponds to a 4×4, and FIG. 9 corresponds to a 2×2 quantization matrix. Here, since the 4×4 quantization matrix is the target, the quantization matrix encoding unit 113 uses the scanning method shown in FIG. 9(b) to arrange the two-dimensional difference value matrix in FIG. 10(a) into the one-dimensional difference value matrix shown in FIG. 10(b). Then, the quantization matrix encoding unit 113 encodes each difference value in FIG. 10(b). Note that although the absolute value of each difference value in FIG. 10(b) is below "1", there may also be cases where the absolute value is greater than 1.By further obtaining the difference between adjacent difference values in consideration of such cases, it is possible to further reduce the amount of code.

[0046] Also, when using the in - quantization - matrix differential coding mode, the quantization - matrix coding unit 113 scans each element of the quantization matrix to be coded using any of FIGS. 9(a) to (c), calculates the difference between two consecutive elements, and arranges them in a one - dimensional matrix. Then, the quantization - matrix coding unit 113 codes each difference value arranged in the one - dimensional matrix as differential information. This corresponds to scaling_list_delta_coef in the syntax table of FIG. 12. For example, since the quantization matrix 806 in FIG. 8(g) is the first to be coded among the quantization matrices with a 4×4 pixel size, there is no quantization matrix to be referred to using the aforementioned quantization - matrix reference mode or the inter - quantization - matrix differential coding mode. Therefore, when coding the quantization matrix 806 in FIG. 8(g), the quantization - matrix coding unit 113 inevitably selects the in - quantization - matrix differential coding mode. Thus, the quantization - matrix coding unit 113 scans the quantization matrix 806 in FIG. 8(g) using FIG. 9(b), and calculates the difference between each element and the previous element in the scanning order. The quantization - matrix coding unit 113 arranges the calculated differences in a one - dimensional matrix to obtain the one - dimensional difference - value matrix shown in FIG. 10(c). Here, for example, the 4×4 - pixel - size quantization matrix in FIG. 8(g) is scanned by the scanning method shown in FIG. 9(b). After the first element 6 located at the upper left, the element 13 located immediately below it is scanned, and the difference +7 is calculated. Also, for coding the first element of the quantization matrix (6 in this embodiment), it is assumed that the difference from a predetermined initial value (for example, 8) is calculated, but of course, it is not limited to this, and the difference from any value or the value of the first element itself may be used. In short, it only needs to be the same initial value as that of the decoding device. Then, the quantization - matrix coding unit 113 codes each difference value in FIG. 10(c).

[0047] Here, a supplementary explanation will be given regarding the encoding process of the quantization matrix described above. The fifth "scaling_list_pred_mode_idx[sizeId][matrixId]" from the top in FIG. 12 is a parameter indicating the encoding mode of the quantization matrix corresponding to sizeId and matrixId. To calculate this parameter, the quantization matrix specified by sizeId and matrixId (any one of FIGS. 8(a) to (l)) is encoded according to the above three encoding modes, and the encoding mode that results in the smallest amount of generated code is determined. Then, information specifying the determined encoding mode (0 to 2 above) is set in this parameter.

[0048] The quantization matrix encoding unit 113 generates quantization matrix code data as the encoding result of each quantization matrix generated using each quantization matrix encoding mode as described above. The quantization matrix encoding unit 113 of the present embodiment encodes by assigning a codeword, which is a binary code, to each element of the one-dimensional difference matrix using the encoding table shown in FIG. 11(a), thereby generating quantization matrix code data. Note that the encoding table is not limited to this, and for example, the encoding table shown in FIG. 11(b) may be used. In this way, the quantization matrix encoding unit 113 outputs the generated quantization matrix code data to the subsequent integrated encoding unit 111.

[0049] Returning to FIG. 1, the integrated encoding unit 111 integrates the code data of the quantization matrix into the header information necessary for encoding the image data.

[0050] Subsequently, the image data is encoded. The image data for one frame input from the input terminal 101 is supplied to the block division unit 102.

[0051] The block division unit 102 divides the input image data for one frame into a plurality of basic blocks, and outputs the image data in units of basic blocks to the prediction unit 104. In the present embodiment, the image data in units of basic blocks of 8×8 pixels is supplied to the prediction unit 104.

[0052] The prediction unit 104 performs prediction processing on the image data in units of basic blocks input from the block division unit 102. Specifically, it determines sub-block division for further dividing a basic block into finer sub-blocks, and further determines prediction modes such as intra prediction and inter prediction in units of sub-blocks. Intra prediction generates predicted pixels of the block to be encoded using the encoded pixels spatially adjacent to the block to be encoded, and also generates an intra prediction mode indicating an intra prediction method such as horizontal prediction, vertical prediction, or DC prediction. Inter prediction generates predicted pixels of the block to be encoded using the encoded pixels of a frame that is temporally different from the block to be encoded, and also generates motion information indicating the frame to be referenced and the motion vector, etc.

[0053] The sub-block division method will be described with reference to FIG. 7. The thick frames of the blocks 700 to 705 in FIGS. 7(a) to 7(f) are the same size as the basic block, which is 8×8 pixels. Each square within the thick frame represents a sub-block. FIG. 7(b) shows an example of conventional square sub-block division, where the 8×8 pixel basic block 701 is divided into 4 sub-blocks of 4×4 pixels. On the other hand, FIGS. 7(c) to 7(f) show an example of rectangular sub-block division. FIG. 7(c) shows that the basic block 702 is divided into 2 sub-blocks of 4×8 pixel size (longitudinal in the vertical direction). FIG. 7(d) shows that the basic block 703 is divided into 2 sub-blocks of 8×4 pixel size (longitudinal in the horizontal direction). FIGS. 7(e) and 7(f) show that, in the case of the basic blocks 704 and 705, although the division methods are different, they are divided into 3 rectangular sub-blocks in a ratio of 1:2:1. Thus, not only squares but also rectangular sub-blocks are used for encoding processing.

[0054] In this embodiment, it will be described that either Fig. 7(a) in which a basic block with a pixel size of 8×8 is not divided into sub-blocks or Fig. 7(b) in which the sub-blocks are divided into a quadtree is used. However, the sub-block division method is not limited to this. A ternary tree division such as in Figs. 7(e) and 7(f) or a binary tree division such as in Figs. 7(c) and 7(d) may be used. When sub-block division other than Figs. 7(a) and 7(b) is also used, a quantization matrix corresponding to the sub-block used in the quantization matrix holding unit 103 is generated. Further, the generated quantization matrix will be encoded by the quantization matrix encoding unit 113.

[0055] The prediction unit 104 generates prediction image data from the determined prediction mode and the encoded region stored in the frame memory 108, further calculates a pixel unit prediction error from the prediction image data corresponding to the target sub-block in the input image data, and outputs the error to the conversion / quantization unit 105. Further, the prediction unit 104 outputs information such as sub-block division and prediction mode as prediction information to the encoding unit 110 and the image reproduction unit 107.

[0056] The transformation and quantization unit 105 performs orthogonal transformation and quantization on the prediction error input from the prediction unit 104 to generate residual coefficients. Specifically, the transformation and quantization unit 105 first performs an orthogonal transformation process corresponding to the size of the sub-block on the prediction error to generate orthogonal transformation coefficients. Then, the transformation and quantization unit 105 quantizes the orthogonal transformation coefficients using the quantization matrix stored in the quantization matrix holding unit 103 according to the prediction mode and color component to generate residual coefficients. In this embodiment, when no sub-block division is performed and the intra prediction mode is used, the quantization matrix in FIG. 8(a) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in FIG. 8(b) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in FIG. 8(c) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when no sub-block division is performed and the inter prediction mode is used instead, the quantization matrix in FIG. 8(d) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in FIG. 8(e) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in FIG. 8(f) is used for the orthogonal transformation coefficients of the Cr component. On the other hand, when the sub-block division in FIG. 7(b) is performed and the intra prediction mode is used, the quantization matrix in FIG. 8(g) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in FIG. 8(h) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in FIG. 8(i) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the sub-block division in FIG. 7(b) is performed and the inter prediction mode is used instead, the quantization matrix in FIG. 8(j) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in FIG. 8(k) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in FIG. 8(l) is used for the orthogonal transformation coefficients of the Cr component. However, the quantization matrix used is not limited to this. The generated residual coefficients and color difference integration information are output to the encoding unit 110 and the inverse quantization and inverse transformation unit 106.

[0057] The inverse quantization and inverse transformation unit 106 reproduces the orthogonal transformation coefficients by inverse quantizing the residual coefficients input from the transformation and quantization unit 105 using the corresponding quantization matrix stored in the quantization matrix holding unit 103. The inverse quantization and inverse transformation unit 106 further performs an inverse orthogonal transformation on the reproduced orthogonal transformation coefficients to reproduce the prediction error. For the inverse quantization process, similar to the transformation and quantization unit 105, a quantization matrix corresponding to the size of the sub-block to be coded and the color component is used. Specifically, the inverse quantization and inverse transformation unit 106 performs inverse quantization using the same quantization matrix as that used by the transformation and quantization unit 105. That is, when the sub-block division is not performed and the intra prediction mode is used, the quantization matrix in Fig. 8(a) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(b) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(c) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the sub-block division is not performed and the inter prediction mode is used instead, the quantization matrix in Fig. 8(d) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(e) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(f) is used for the orthogonal transformation coefficients of the Cr component. On the other hand, when the sub-block division in Fig. 7(b) is performed and the intra prediction mode is used, the quantization matrix in Fig. 8(g) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(h) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(i) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the sub-block division in Fig. 7(b) is performed and the inter prediction mode is used instead, the quantization matrix in Fig. 8(j) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(k) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(l) is used for the orthogonal transformation coefficients of the Cr component.

[0058] The prediction error reproduced by performing an inverse orthogonal transformation on the thus reproduced orthogonal transformation coefficients is output to the image reproduction unit 107.

[0059] The image reproduction unit 107 appropriately refers to the frame memory 108 based on the prediction information input from the prediction unit 104 and reproduces the predicted image. Then, the image reproduction unit 107 generates the reproduced image data in sub-block units based on the reproduced predicted image and the prediction error of the sub-block reproduced by the inverse quantization and inverse transformation unit 106, and stores it in the frame memory 108.

[0060] The in-loop filter unit 109 reads the reproduced image data from the frame memory 108 and performs in-loop filter processing such as deblocking filtering. Then, the in-loop filter unit 109 stores the filtered image data back into the frame memory 108.

[0061] The encoding unit 110 performs entropy encoding on the residual coefficients and color difference integration information in sub-block units generated by the transformation / quantization unit 105, as well as the prediction information input from the prediction unit 104, to generate coded data. Although the method of entropy encoding is not particularly specified, Golomb encoding, arithmetic encoding, Huffman encoding, etc. can be used. The encoding unit 110 outputs the generated coded data to the integrated encoding unit 111.

[0062] The integrated encoding unit 111 multiplexes the coded data input from the encoding unit 110 together with the coded data of the above-mentioned header to form a bitstream. Then, the integrated encoding unit 111 outputs the formed bitstream from the output terminal 112 to the outside (such as a storage medium or a network).

[0063] FIG. 6(a) is an example of the data structure of the bitstream output in this embodiment. The sequence header includes the coded data of the quantization matrix and is composed of the encoding results of each quantization matrix. However, the position where the coded data of the quantization matrix is stored is not limited to this, and it may of course have a configuration arranged in a picture header part such as that in FIG. 6(b) or a header part spanning a plurality of pictures. Also, when changing the quantization matrix within one sequence, it is also possible to update by newly encoding the quantization matrix. At this time, all the quantization matrices may be rewritten, or a part of them may be changed by specifying the size, prediction mode, and color component of the quantization matrix corresponding to the quantization matrix to be rewritten.

[0064] FIG. 3 is a flowchart showing the encoding process for one frame of the control unit 150 in the image encoding apparatus according to the embodiment.

[0065] First, prior to encoding the image, at S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold a two-dimensional quantization matrix. The quantization matrix holding unit 103 according to this embodiment corresponds to blocks of 8×8 to 2×2 pixel sizes, and generates and holds quantization matrices corresponding to the respective color components and prediction modes shown in FIGS. 8(a) to (l).

[0066] At S302, the control unit 150 controls the quantization matrix encoding unit 113 to encode the quantization matrix generated and held at S301. Since the specific operation of the quantization matrix encoding unit 113 here has been described, it is omitted. In this embodiment, the control unit 150 controls the quantization matrix encoding unit 113 to perform encoding on the quantization matrices 801 to 812 shown in FIGS. 8(a) to (l) based on the syntax table of FIG. 12, and generate quantization matrix encoded data.

[0067] At S303, the control unit 150 controls the integrated encoding unit 111 to encode and output the generated quantization matrix encoded data together with the header information necessary for encoding the image data.

[0068] At S304, the control unit 150 controls the block division unit 102 to divide the input image in frame units into basic block units.

[0069] At S305, the control unit 150 controls the prediction unit 104 to perform division of the image data in basic block units generated at S304 into sub-blocks and prediction processing for each sub-block, and generate prediction information such as sub-block division information and prediction mode, and prediction image data. Further, the control unit 150 controls the prediction unit 104 to calculate a prediction error from the input sub-block image data and the prediction image data.

[0070] In S306, the control unit 150 controls the conversion / quantization unit 105 to perform orthogonal transformation on the prediction error calculated in S305 to generate orthogonal transformation coefficients. Further, the control unit 150 controls the conversion / quantization unit 105 to perform quantization using the quantization matrix generated and held in S301 to generate residual coefficients. In this embodiment, it is assumed that the quantization matrices in FIGS. 8(a) to 8(l) are used according to the size of the sub-block, the prediction mode, and the color component.

[0071] In S307, the control unit 150 controls the inverse quantization / inverse transformation unit 106 to perform inverse quantization on the residual coefficients generated in S306 using the quantization matrix generated and held in S301 to reproduce the orthogonal transformation coefficients. In this step, the same quantization matrix as that used in S306 is used, and inverse quantization processing is performed. Then, inverse orthogonal transformation is performed on the reproduced orthogonal transformation coefficients to reproduce the prediction error.

[0072] In S308, the control unit 150 controls the image reproduction unit 107 to reproduce a predicted image based on the prediction information generated in S305, reproduce image data from the reproduced predicted image and the prediction error generated in S307, and store it in the frame memory 108.

[0073] In S309, the control unit 150 controls the encoding unit 110 to perform encoding of the prediction information generated in S305 and the residual coefficients generated in S306 to generate encoded data. Further, the encoding unit 110 outputs the generated encoded data to the integrated encoding unit 111. The integrated encoding unit 111 positions the encoded data from the encoding unit 110 to follow the previously generated header and outputs it.

[0074] In S310, the control unit 150 determines whether or not the encoding of all the basic blocks within the target frame has been completed. If the control unit 150 determines that it has been completed, it proceeds to S311. If it determines that there are still uncoded basic blocks remaining, it returns the process to S304 to continue encoding for the next basic block.

[0075] In S311, the control unit 150 controls the in-loop filter unit 109, performs in-loop filter processing on the image data reproduced in S308, generates a filtered image, and ends the process.

[0076] With the above configuration and operation, particularly in S302, by determining and encoding the quantization matrix encoding mode in which the amount of generated codes by the quantization matrix is minimized, it is possible to suppress the amount of generated codes by the quantization matrix.

[0077] In this embodiment, in the quantization matrix reference mode and the quantization matrix differential encoding mode, only quantization matrices of the same size can be referenced. However, a configuration may be adopted in which quantization matrices of different sizes can be referenced using enlargement / reduction. For example, when encoding the quantization matrix 806 in FIG. 8(g), it may be configured to enlarge with reference to the quantization matrix 807 in FIG. 8(h) and encode the difference value between the two as differential information. Thereby, it is also possible to further reduce the data amount of the quantization matrix encoded first in each size.

[0078] Also, in this embodiment, as shown in the syntax table of FIG. 12, encoding is performed in order from the smaller quantization matrix, but a configuration may be adopted in which encoding is performed in order from the larger quantization matrix. Thereby, particularly when reference between quantization matrices of different sizes as described above is enabled, it is possible to further reduce the data amount of the quantization matrix.

[0079] Furthermore, in this embodiment, the reference target of the quantization matrix is limited to the encoded quantization matrix, but it is also possible to use a flat quantization matrix that serves as a reference when no quantization matrix is used, as shown in FIGS. 13(a) to (c), as the reference target. Thereby, particularly when the quantization matrix to be encoded is close to a flat quantization matrix, it is possible to further reduce the data amount of the quantization matrix.

[0080] FIG. 2 is a block diagram of an image decoding apparatus that decodes the encoded image data generated by the above image encoding apparatus. Hereinafter, with reference to the same figure, the configuration related to the decoding process and its operation will be described.

[0081] The image decoding apparatus has a control unit 250 that controls the entire apparatus. This control unit 250 has a CPU, a ROM that stores programs executed by the CPU, and a RAM used as a work area for the CPU. Further, the image decoding apparatus has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization / inverse transformation unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0082] The input terminal 201 inputs an encoded bit stream, and the input source is, for example, a storage medium storing an encoded stream, but it may also be input from a network, and its type is not limited.

[0083] The separation decoding unit 202 separates the information related to the decoding process and the coded data related to the coefficients from the bit stream, and also decodes the coded data existing in the header part of the bit stream. The separation decoding unit 202 of the present embodiment separates the quantization matrix coded data and outputs it to the quantization matrix decoding unit 209. Further, the separation decoding unit 202 outputs the coded data of the image to the decoding unit 203. That is, the separation decoding unit 202 performs an operation opposite to that of the integrated encoding unit 111 in FIG. 1.

[0084] The quantization matrix decoding unit 209 decodes the quantization matrix coded data supplied from the separation decoding unit 202 to reproduce and hold the quantization matrix.

[0085] The decoding unit 203 decodes the coded data of the image output from the separation decoding unit 202 and reproduces the residual coefficients and prediction information in sub-block units.

[0086] The inverse quantization and inverse transformation unit 204, similar to the inverse quantization and inverse transformation unit 106 in FIG. 1, performs inverse quantization on the residual coefficients of the target sub-block using the reproduced quantization matrix to obtain the coefficients after inverse quantization, and further performs an inverse orthogonal transformation to reproduce the prediction error.

[0087] The image reproduction unit 205 generates prediction image data by appropriately referring to the frame memory 206 based on the input prediction information. Then, the image reproduction unit 205 generates the reproduced image data of the target sub-block from this prediction image data and the prediction error reproduced by the inverse quantization and inverse transformation unit 204, and stores it in the frame memory 206.

[0088] The in-loop filter unit 207, similar to the in-loop filter unit 109 in FIG. 1, performs in-loop filter processing such as a deblocking filter on the reproduced image data stored in the frame memory 206, and stores the image data after the filter processing back in the frame memory 206.

[0089] The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. Although the output destination is generally a display device, it may be other devices.

[0090] The operation related to the decoding of the image of the image decoding device according to the above embodiment will be described in more detail. In this embodiment, the configured is such that the encoded bit stream is input in units of frames.

[0091] In FIG. 2, a bit stream for one frame input from the input terminal 201 is supplied to the separation and decoding unit 202. The separation and decoding unit 202 separates information related to the decoding process and coded data related to coefficients from the bit stream, and decodes the coded data present in the header portion of the bit stream. Then, the separation and decoding unit 202 supplies the quantization matrix coded data included in the header portion to the quantization matrix decoding unit 209, and supplies the coded data of the image data to the decoding unit 203. Specifically, the separation and decoding unit 202 first extracts the quantization matrix coded data from the sequence header of the bit stream shown in FIG. 6(a), and outputs the extracted quantization matrix coded data to the quantization matrix decoding unit 209. In this embodiment, the quantization matrix coded data corresponding to the quantization matrices shown in FIGS. 8(a) to (l) is extracted and output. Subsequently, the coded data of the picture data in units of basic blocks is extracted and output to the decoding unit 203.

[0092] The quantization matrix decoding unit 209 first decodes the input quantization matrix coded data, and reproduces the coding results of the respective quantization matrices generated on the coding side based on the syntax table shown in FIG. 12. As shown in FIG. 6(a), the coding result of each quantization matrix is composed of a quantization matrix coding mode and data corresponding to each quantization matrix coding mode.

[0093] For example, in the case of the quantization matrix encoding mode, that is, when the scaling_list_pred_mode_idx in the syntax table of FIG. 12 is 0, it means that the quantization matrix is encoded in the quantization matrix reference mode. In this case, the encoding result of the quantization matrix is composed of the quantization matrix encoding mode (0) and the quantization matrix reference index. For example, when decoding the 4×4 quantization matrix for intra prediction·Cb component, the quantization matrix decoding unit 209 first reproduces the quantization matrix encoding mode = 0, which means the quantization matrix reference mode. Then, the quantization matrix decoding unit 209 reproduces the quantization matrix reference index. When the quantization matrix reference index indicates that the quantization matrix to be referenced among the decoded quantization matrices was decoded immediately before, the quantization matrix reference index = 0. When the quantization matrix reference index = 1, the quantization matrix decoded one more time before is referenced. Based on this information, the quantization matrix decoding unit 209 can find that the 4×4 quantization matrix for intra prediction·Cb component is the same as the quantization matrix 806 in FIG. 8(g) for intra prediction·Y component, and reproduces the quantization matrix 801 in FIG. 8(b).

[0094] Also, in the case of the quantization matrix encoding mode, that is, when scaling_list_pred_mode_idx in the syntax table of FIG. 12 is 1, it means that the quantization matrix is encoded in the quantization matrix inter-difference encoding mode. In this case, the encoding result of the quantization matrix is composed of the quantization matrix encoding mode (1), the quantization matrix reference index, and the difference information. For example, when decoding a 4×4 quantization matrix for intra prediction and the Cr component, the quantization matrix decoding unit 209 first reproduces the quantization matrix encoding mode = 1, which means the quantization matrix inter-difference encoding mode. Then, the quantization matrix decoding unit 209 reproduces the quantization matrix reference index. When the quantization matrix reference index indicates that the quantization matrix to be referenced among the decoded quantization matrices was decoded immediately before, the quantization matrix reference index = 0. When the quantization matrix reference index = 1, the quantization matrix decoded one more time before is further referenced. Subsequently, the quantization matrix decoding unit 209 decodes the difference information, that is, scaling_list_pred_delta in the syntax table of FIG. 12, for the number of elements of the quantization matrix, and reproduces the one-dimensional difference matrix shown in FIG. 10(b). The quantization matrix decoding unit 209 arranges the one-dimensional difference matrix in FIG. 10(b) two-dimensionally using the scanning method in FIG. 9(b) to obtain the two-dimensional difference matrix shown in FIG. 10(a). Then, the quantization matrix decoding unit 209 adds each difference value of the two-dimensional difference matrix in FIG. 10(a) to each element of the quantization matrix 801 in FIG. 8(b) that is the reference target, and reproduces the quantization matrix 802 in FIG. 8(c).

[0095] On the one hand, in the case of the quantization matrix encoding mode, that is, when the scaling_list_pred_mode_idx in the syntax table of FIG. 12 is 2, it means that the quantization matrix has been encoded in the in-quantization matrix differential encoding mode. In this case, the encoding result of the quantization matrix is composed of the quantization matrix encoding mode (2) and the differential information. For example, when decoding a 4×4 quantization matrix for intra prediction·Y component, the quantization matrix decoding unit 209 first reproduces the quantization matrix encoding mode = 2, which means the in-quantization matrix differential encoding mode. Then, the quantization matrix decoding unit 209 decodes the differential information, that is, the scaling_list_delta_coef in the syntax table of FIG. 12, for the number of elements of the quantization matrix, and reproduces the one-dimensional difference matrix shown in FIG. 10(c). The quantization matrix decoding unit 209 arranges it two-dimensionally using the scanning method of FIG. 9(b) while adding each difference value in the one-dimensional difference matrix of FIG. 10(b) to the previous element, and reproduces the quantization matrix 806 of FIG. 8(g).

[0096] In this embodiment, it is assumed that the encoding table of FIG. 11(a) is used for decoding the differential information, but the encoding table of FIG. 11(b) may also be used. In short, the same one as the encoding side may be used. Then, the quantization matrix decoding unit 209 holds the quantization matrices 800 to 811 reproduced in this way. Here, an operation reverse to that of the quantization matrix encoding unit 113 on the encoding side is performed.

[0097] The decoding unit 203 decodes the coded data supplied from the separation decoding unit 202, reproduces the prediction information, and further reproduces the residual coefficients. First, the decoding unit 203 reproduces the prediction information and obtains the prediction mode used in the sub-block. The decoding unit 203 outputs the reproduced residual coefficients to the inverse quantization and inverse transformation unit 204, and outputs the reproduced prediction information to the image reproduction unit 205.

[0098] The inverse quantization and inverse transformation unit 204 performs inverse quantization on the input residual coefficients using the quantization matrix reproduced by the quantization matrix decoding unit 209 to generate orthogonal transformation coefficients, and further performs inverse orthogonal transformation to reproduce the prediction error. Similar to the inverse quantization and inverse transformation unit 106 on the encoding side, the inverse quantization and inverse transformation unit 204 performs inverse quantization using a quantization matrix corresponding to the size of the sub-block to be decoded and the color component. That is, when the intra prediction mode is used without sub-block division, the quantization matrix in Fig. 8(a) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(b) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(c) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the inter prediction mode is used instead without sub-block division, the quantization matrix in Fig. 8(d) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(e) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(f) is used for the orthogonal transformation coefficients of the Cr component. On the other hand, when the sub-block division in Fig. 7(b) is performed and the intra prediction mode is used, the quantization matrix in Fig. 8(g) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(h) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(i) is used for the orthogonal transformation coefficients of the Cr component. Similarly, when the sub-block division in Fig. 7(b) is performed and the inter prediction mode is used instead, the quantization matrix in Fig. 8(j) is used for the orthogonal transformation coefficients of the Y component, the quantization matrix in Fig. 8(k) is used for the orthogonal transformation coefficients of the Cb component, and the quantization matrix in Fig. 8(l) is used for the orthogonal transformation coefficients of the Cr component.

[0099] The prediction error reproduced by performing inverse orthogonal transformation on the orthogonal transformation coefficients reproduced in this way is output to the image reproduction unit 205. However, the quantization matrix used is not limited to this, and it may be the same as the quantization matrix used in the transformation and quantization unit 105 and the inverse quantization and inverse transformation unit 106 on the encoding side.

[0100] The image reproduction unit 205 appropriately refers to the frame memory 206 based on the prediction information input from the decoding unit 203, and reproduces a predicted image. The image reproduction unit 205 of the present embodiment uses intra prediction and inter prediction, similar to the prediction unit 104 on the encoding side. Since the specific prediction process is the same as that of the prediction unit 104 on the encoding side, the description thereof is omitted. The image reproduction unit 205 reproduces image data from this predicted image and the prediction error input from the inverse quantization / inverse transformation unit 204, and stores it in the frame memory 206. The stored image data is used for reference during prediction.

[0101] The in-loop filter unit 207 reads the reproduced image from the frame memory 206 and performs in-loop filter processing such as a deblocking filter, similar to the in-loop filter unit 109 on the encoding side. Then, the in-loop filter unit 207 stores the filtered image back in the frame memory 206.

[0102] The reproduced image stored in the frame memory 206 is finally output to the outside (represented by a display device) from the output terminal 208.

[0103] FIG. 4 is a flowchart showing the decoding process of the control unit 250 in the image decoding apparatus according to the embodiment.

[0104] First, at S401, the control unit 250 controls the separation decoding unit 202 to separate the information related to the decoding process and the coded data related to the coefficients from the bit stream, and decodes the coded data in the header part. More specifically, the separation decoding unit 202 supplies the coded data of the quantization matrix to the quantization matrix decoding unit 209, and supplies the coded data of the image to the decoding unit 203.

[0105] In S402, the control unit 250 controls the quantization matrix decoder 209 to decode the quantization matrix coded data reproduced in S401 based on the syntax table in FIG. 12, and reproduce and hold the quantization matrices 801 to 811 shown in FIGS. 8(a) to (l). Since the specific operation of the quantization matrix decoder 209 here has been described, it is omitted.

[0106] In S403, the control unit 250 controls the decoder 203 to decode the coded data separated in S401, reproduce the prediction information, and reproduce the residual coefficients.

[0107] In S404, the control unit 250 controls the inverse quantization / inverse transformation unit 204 to perform inverse quantization on the residual coefficients using the quantization matrix reproduced in S402 to generate orthogonal transformation coefficients. The inverse quantization / inverse transformation unit 204 further performs inverse orthogonal transformation to reproduce the prediction error. In this embodiment, the quantization matrix used in the inverse quantization process is determined according to the color component and size of the sub-block to be decoded. That is, the inverse quantization / inverse transformation unit 204 performs inverse quantization using the quantization matrices in FIGS. 8(a) to 8(l) according to the size, prediction mode, and color component of the sub-block. However, the quantization matrix used is not limited to this, and it may be the same as the quantization matrix used on the encoding side.

[0108] In S405, the control unit 250 controls the image reproduction unit 205 to reproduce an image based on the prediction information generated in S403. Specifically, the image reproduction unit 205 reproduces a predicted image by referring to the frame memory 206 based on the prediction information. At this time, the image reproduction unit 205 uses intra prediction and inter prediction in the same manner as S305 on the encoding side. Then, the image reproduction unit 205 reproduces image data from the reproduced predicted image and the prediction error generated in S404, and stores the reproduced image data in the frame memory 206.

[0109] In S406, the control unit 250 determines whether the decoding of all the basic blocks within the target frame has been completed. If it has been completed, the process proceeds to S407. If there are unsigned basic blocks, the process returns to S403 to perform the process of setting the next basic block as the decoding target.

[0110] In S407, the control unit 250 controls the in-loop filter unit 207 to perform in-loop filter processing on the image data reproduced in S405, generate a filtered image, and end the process.

[0111] With the above configuration and operation, it is possible to decode the encoded bitstream generated by the image encoding device described above, that is, the bitstream encoded in an appropriate quantization matrix encoding mode and suppressing the amount of generated code by the quantization matrix.

[0112] Note that in this embodiment, in the quantization matrix reference mode and the quantization matrix differential encoding mode, only quantization matrices of the same size can be referenced. However, a configuration may be adopted such that quantization matrices of different sizes can be referenced using enlargement and reduction. For example, when decoding the quantization matrix 806 in FIG. 8(g), it may be configured to refer to the quantization matrix 807 in FIG. 8(h), enlarge it, decode the difference value between the two as differential information, and reproduce the quantization matrix. Thereby, it is also possible to decode a bitstream with an even further reduced data amount of the quantization matrix encoded first in each size.

[0113] Also, in this embodiment, as shown in the syntax table of FIG. 12, the configuration is such that decoding is performed in order from the smallest quantization matrix, but a configuration may be adopted such that decoding is performed in order from the largest quantization matrix. Thereby, especially when referencing between quantization matrices of different sizes as described above, it becomes possible to decode a bitstream with an even further reduced data amount of the quantization matrix.

[0114] Furthermore, in this embodiment, the reference target of the quantization matrix is limited to the decoded quantization matrix. However, as shown in FIGS. 13(a) to (c), a flat quantization matrix that serves as a reference when not using a quantization matrix can also be used as the reference target. As a result, particularly when the quantization matrix to be decoded is close to a flat quantization matrix, it becomes possible to decode a bit stream in which the data amount of the quantization matrix is further reduced.

[0115] [Second Embodiment] The processing units included in the image encoding device and the image decoding device in the above first embodiment have been described as being configured by hardware. However, the processing performed by each of the processing units shown in these figures may be configured by a computer program.

[0116] FIG. 5 is a block diagram showing a configuration example of the hardware of a computer applicable to the image encoding device and the decoding device according to the above embodiment.

[0117] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and the ROM 503, and executes each of the processes described above as being performed by the image processing device according to the above embodiment. That is, the CPU 501 functions as each of the processing units shown in FIGS. 1 and 2.

[0118] The RAM 502 has an area for temporarily storing data acquired from the outside via the external storage device 506 and the I / F (interface) 507. Further, the RAM 502 is also used as a work area used when the CPU 501 executes various processes. The RAM 502 can be allocated, for example, as a frame memory, or can appropriately provide various other areas.

[0119] The ROM 503 stores the configuration data of this computer, the boot program, etc. The operation unit 504 is composed of a keyboard, a mouse, etc., and various instructions can be input to the CPU 501 by the user of this computer operating it. The display unit 505 displays the processing results by the CPU 501. Also, the display unit 505 is composed of, for example, a liquid crystal display.

[0120] The external storage device 506 is a large-capacity information storage device typified by a hard disk drive device. The external storage device 506 stores an OS (Operating System) and computer programs (application programs) for causing the CPU 501 to realize the functions of each part shown in FIGS. 1 and 2. Furthermore, each image data to be processed may be stored in the external storage device 506.

[0121] The computer programs and data stored in the external storage device 506 are appropriately loaded into the RAM 502 according to the control by the CPU 501 and become the processing targets by the CPU 501. The I / F 507 can connect other devices such as a network such as a LAN or the Internet, a projection device, and a display device, and this computer can acquire and send out various information via this I / F 507. 508 is a bus connecting the above-mentioned respective parts.

[0122] In the above configuration, when the power is turned on to this device, the CPU 501 executes the boot program stored in the ROM 503, loads and executes the OS stored in the external storage device 506 into the RAM 502. Then, under the control of the OS, the CPU 501 loads and executes the application program related to encoding or decoding from the external storage device 506 into the RAM 502. As a result, the CPU 501 functions as each processing unit in FIG. 1 or FIG. 2, and this device functions as an image encoding device or an image decoding device.

[0123] Here, the encoding process of the quantization matrix in the second embodiment, that is, the process corresponding to S302 in FIG. 3, will be described with reference to the flowchart in FIG. 14.

[0124] Note that the order of the quantization matrices to be encoded is the same as that in the first embodiment. First, in the order of quantization matrices 807→808→810→811 with a 2×2 pixel size, then, in the order of quantization matrices 806→801→802→809→804→806 with a 4×4 pixel size, and finally, in the order of quantization matrices 800→803 with an 8×8 pixel size.

[0125] Also, the variables sizeId and matrixId shown below have the same meaning as those shown in the first embodiment.

[0126] That is, the variable sizeId is a variable indicating the size of the quantization matrix. When sizeId is "1", it indicates that the size of the quantization matrix to be encoded is 2×2 pixel size. When sizeId is "2", it indicates that the size of the quantization matrix to be encoded is 4×4 pixel size. When sizeId is "3", it indicates that the size of the quantization matrix to be encoded is 8×8 pixel size. Also, the variable matrixId indicates the order of the quantization matrices of the same size. And the quantization matrix to be encoded is specified by sizeId and matrixId.

[0127] For example, when sizeId = 2 and matrixId = 0, it represents that the first quantization matrix 806 with a 4×4 pixel size is the quantization matrix to be encoded.

[0128] First, in S1401, the CPU 501 sets the size of the quantization matrix to be encoded to a 2×2 pixel size by setting the variable sizeId to “1” as the initial value. Next, in S1402, the CPU 501 sets the matrixId to “0” as the initial value. As a result, when this S1402 is executed for the first time, the quantization matrix 807 in FIG. 8 is set as the quantization matrix of interest to be encoded.

[0129] In S1403, the CPU 501 determines whether the matrixId is “0”. If the matrixId is other than “0”, there will be an encoded quantization matrix of the same size as the quantization matrix of interest. Therefore, the CPU 501 advances the process to S1404 and performs a search process in the quantization matrix reference mode. That is, the CPU 501 performs a search process to determine whether there is an identical one in the encoded quantization matrices as the quantization matrix of interest.

[0130] In S1405, the CPU 501 determines whether it has been able to find an encoded quantization matrix that is the same as the quantization matrix of interest. If it is determined that it has been found, the CPU 501 advances the process to S1406 and outputs “0” and the value identifying the matching quantization matrix as the encoded data of the quantization matrix of interest. Setting “0” at the beginning of the encoded data is equivalent to setting the parameter of scaling_list_pred_mode_idx to “0”. Also, in the case of the embodiment, since the number of quantization matrices of 4×4 pixel size “6” is larger than the number of quantization matrices of other sizes, it is sufficient to have 3 bits for the value assigned to identify the quantization matrix. On the other hand, if it is determined that it has not been found, the CPU 501 advances the process to S1407.

[0131] In S1403, when the CPU 501 determines that the matrixId is “0”, the process advances to S1409.

[0132] In S1407, the CPU 501 executes the inter-quantization matrix differential encoding mode for the target quantization matrix. Then, in S1408, the CPU 501 obtains "1" and the encoded data by the inter-quantization matrix differential encoding mode. Setting "1" at the beginning of the encoded data is equivalent to setting the parameter of scaling_list_pred_mode_idx to "1". Information indicating the order of the quantization matrix identical to the target quantization matrix may be output following the leading "1", and then the differential encoded data may be output.

[0133] Next, in S1409, the CPU 501 executes the intra-quantization matrix differential encoding mode for the target quantization matrix. Then, in S1410, the CPU 501 obtains "2" and the encoded data by the inter-quantization matrix differential encoding mode. Setting "2" at the beginning of the encoded data is equivalent to setting the return value of scaling_list_pred_mode_idx to "2".

[0134] Then, in S1411, the CPU 501 outputs the smaller of the encoded data generated in S1408 and S1410 as the encoded data of the target quantization matrix. When branching from S1403 to S1409, the encoded data generated by the intra-quantization matrix differential encoding mode in S1409 is output.

[0135] In S1412, the CPU 501 increments the variable matrixId by "1". Then, in S1413, the CPU 501 determines whether encoding of all quantization matrices of the same size as the target quantization matrix has been completed based on the variables sizeId and matrixId. If the determination result in S1413 is "No", the CPU 501 advances the process to S1403 and repeats the above process.

[0136] Also, when the determination result of S1413 is “Yes”, the CPU 501 increments sizeId by “1”. Then, in S1415, the CPU 501 determines whether the encoding of the quantization matrices of all sizes has been completed based on the variable sizeId and matrixId. If the determination result of S1415 is “No”, it means that there are quantization matrices of sizes that have not been encoded, so the CPU returns the process to S1402. If the determination result of S1415 is “Yes”, this process (S302 in FIG. 3) ends.

[0137] Note that when the process proceeds to S1406, it is agreed that the amount of encoded data in the quantization matrix reference mode decreases. Therefore, comparison with the encoded data in other modes is omitted, but of course, comparison may also be performed.

[0138] (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and having one or more processors in the computer of the system or apparatus read and execute the program. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Industrial Applicability

[0139] The present invention is used in an encoding device and a decoding device that perform encoding and decoding of still images and moving images. In particular, it is applicable to an encoding method and a decoding method that use a quantization matrix.

Explanation of Signs

[0140] 101... Input terminal, 102... Block division unit, 103... Quantization matrix holding unit, 104... Prediction unit, 105... Transformation / quantization unit, 106... Inverse quantization / inverse transformation unit, 107... Image reproduction unit, 108... Frame memory, 109... In-loop filter unit, 110... Encoding unit, 111... Integrated encoding unit, 112... Output terminal, 113... Quantization matrix encoding unit, 150... Control unit

Claims

1. A decoding means for decoding the first quantization matrix; a dequantization means for dequantizing the residual coefficients using the first quantization matrix; The decoding means decodes first information representing a mode of decoding of the first quantization matrix from a scaling_list_data syntax structure; When the decoded first information indicates that a first mode is used, the decoding means decodes second information for identifying a second quantization matrix to be referenced from the scaling_list_data syntax structure, and derives the first quantization matrix by determining that the second quantization matrix identified based on the second information is identical to the first quantization matrix; If the decoded first information indicates that a second mode is used, the decoding means decodes the second information for identifying the second quantization matrix from the scaling_list_data syntax structure; (a) associating a first element in a first one-dimensional matrix consisting of a plurality of elements, the first element being an element of a difference value between an element in the first row and the first column of the first quantization matrix and an element in the first row and the first column of the second quantization matrix specified based on the second information, with the first row and the first column of a two-dimensional matrix as a first element in a predetermined scanning order, and deriving the element in the first row and the first column of the first quantization matrix using the first element associated with the first row and the first column and the element in the first row and the first column of the second quantization matrix; (b) associating a second element in the first one-dimensional matrix, the second element being a difference value between the element in the second row, the first column of the first quantization matrix and the element in the second row, the first column of the second quantization matrix, with the second row, the first column of a two-dimensional matrix as a second element in the predetermined scanning order, and deriving the element in the second row, the first column of the first quantization matrix using the second element associated with the second row, the first column and the element in the second row, the first column of the second quantization matrix; (c) associating a third element in the first one-dimensional matrix, the third element being a difference value between the element in the first row, the second column of the first quantization matrix and the element in the first row, the second column of the second quantization matrix, with the first row, the second column of a two-dimensional matrix as a third element in the predetermined scanning order, and decoding the first quantization matrix by using the third element associated with the first row, the second column and the element in the first row, the second column of the second quantization matrix to derive the element in the first row, the second column of the first quantization matrix; When the decoded first information indicates that a third mode is used, the decoding means decodes from the scaling_list_data syntax structure a plurality of difference values ​​including a difference value between an element in the first row and first column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order; (a) using a first element in a second one-dimensional matrix obtained from the plurality of difference values, deriving an element in the first row and first column of the first quantization matrix corresponding to a first element in the predetermined scanning order; (b) using a second element in the second one-dimensional matrix, deriving an element in the second row and first column of the first quantization matrix that corresponds to a second element in the predetermined scanning order; (c) using a third element in the second one-dimensional matrix, deriving an element in the first row and second column of the first quantization matrix that corresponds to a third element in the predetermined scanning order; (d) decoding the first quantization matrix using a fourth element in the second one-dimensional matrix to derive an element in the third row and first column of the first quantization matrix that corresponds to the fourth element in the predetermined scanning order; the decoding means is capable of using a quantization matrix in which all elements are set to 16 as the second quantization matrix; The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used.

2. An image decoding device comprising:

2. 2. The image decoding device according to claim 1, wherein the first quantization matrix and the second quantization matrix have the same size.

3. The image decoding device as described in Claim 1, characterized in that the size of the second quantization matrix is ​​only one of 2x2, 4x4, or 8x8.

4. The image decoding device described in Claim 1, characterized in that the amount of code of the first information indicating that the first mode is used is not greater than the amount of code of the first information indicating that the third mode is used.

5. the first element of the predetermined scanning order is the first row and first column of a two-dimensional matrix; the second element of the predetermined scan order is the second row and first column of the two-dimensional matrix; the third element of the predetermined scan order is the first row and second column of the two-dimensional matrix; 2. The image decoding device according to claim 1, wherein the fourth element in the predetermined scanning order is in the third row and first column of a two-dimensional matrix.

6. When the decoded first information indicates that a second mode is to be used, the decoding means a fourth element in the first one-dimensional matrix, the fourth element being an element of a difference value between the element in the third row, the first column of the first quantization matrix and the element in the third row, the first column of the second quantization matrix, is associated with the third row, the first column of a two-dimensional matrix as a fourth element in the predetermined scanning order, and the element in the third row, the first column of the first quantization matrix is ​​derived using the fourth element associated with the third row, the first column and the element in the third row, the first column of the second quantization matrix; 2. The image decoding device according to claim 1, further comprising: a decoding unit for decoding the first quantization matrix.

7. quantization means for quantizing transform coefficients of a block of an image using a first quantization matrix; and encoding means for encoding the first quantization matrix; When a first mode is used to encode the first quantization matrix, the encoding means encodes information indicating that the first quantization matrix is ​​identical to a reference second quantization matrix and information for identifying the second quantization matrix into a scaling_list_data syntax structure; When a second mode is used to code the first quantization matrix, the coding means codes information for identifying the second quantization matrix in the scaling_list_data syntax structure; (a) a first element, which is a first element in a predetermined scanning order, and which is an element of a difference value between an element in a first row and a first column of the first quantization matrix and an element in a first row and a first column of the second quantization matrix; (b) a second element in the predetermined scanning order, the second element being a difference value between an element in a second row and a first column of the first quantization matrix and an element in a second row and a first column of the second quantization matrix; (c) encoding information used to derive a first one-dimensional matrix including a third element in the predetermined scanning order, the third element being an element of a difference value between an element in a first row and a second column of the first quantization matrix and an element in a first row and a second column of the second quantization matrix; When a third mode is used to encode the first quantization matrix, the encoding means (a) encoding, into the scaling_list_data syntax structure, a difference value between an element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order and a predetermined value; (b) encoding, into the scaling_list_data syntax structure, a difference value between an element in the second row and the first column of the first quantization matrix corresponding to a second element in the predetermined scanning order and an element in the first row and the first column of the first quantization matrix; (c) encoding a difference value between an element in the first row and second column of the first quantization matrix corresponding to a third element in the predetermined scanning order and an element in the second row and first column of the first quantization matrix into the scaling_list_data syntax structure; (d) encoding a difference value between an element in the third row and the first column of the first quantization matrix corresponding to a fourth element in the predetermined scanning order and an element in the first row and the second column of the first quantization matrix into the scaling_list_data syntax structure; encoding a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order; the encoding means is capable of using as the second quantization matrix a quantization matrix in which all elements are set to 16; The code amount of the information representing that the first mode is used is smaller than the code amount of the information representing that the second mode is used.

1. An image encoding device comprising:

8. 8. The image encoding device according to claim 7, wherein the first quantization matrix and the second quantization matrix have the same size.

9. The image encoding device as described in Claim 7, characterized in that the size of the second quantization matrix is ​​only one of 2x2, 4x4, or 8x8.

10. An image encoding device as described in Claim 7, characterized in that the amount of code of information representing that the first mode is used is not greater than the amount of code of information representing that the third mode is used.

11. The first element of the predetermined scanning order is the first row and first column of a two-dimensional matrix, the second element of the predetermined scan order is the second row and first column of the two-dimensional matrix; the third element of the predetermined scan order is the first row and second column of the two-dimensional matrix; 8. The image encoding device according to claim 7, wherein the fourth element in the predetermined scanning order is in the third row and first column of a two-dimensional matrix.

12. The image encoding device described in Claim 7, characterized in that the multiple elements include a fourth element which is a fourth element in the specified scanning order and is an element of a difference value between the element in the third row and first column of the first quantization matrix and the element in the third row and first column of the second quantization matrix.

13. a decoding step of decoding the first quantization matrix; and an inverse quantization step of inverse quantizing the residual coefficients using the first quantization matrix, The decoding step includes decoding first information representing a mode of decoding the first quantization matrix from a scaling_list_data syntax structure; When the decoded first information indicates that a first mode is used, in the decoding step, second information for identifying a second quantization matrix to be referenced is decoded from the scaling_list_data syntax structure, and the first quantization matrix is ​​derived by determining that the second quantization matrix identified based on the second information is identical to the first quantization matrix; If the decoded first information indicates that a second mode is used, in the decoding step, decoding the second information for identifying the second quantization matrix from the scaling_list_data syntax structure; (a) associating a first element in a first one-dimensional matrix consisting of a plurality of elements, the first element being an element of a difference value between an element in the first row and the first column of the first quantization matrix and an element in the first row and the first column of the second quantization matrix specified based on the second information, with the first row and the first column of a two-dimensional matrix as a first element in a predetermined scanning order, and deriving the element in the first row and the first column of the first quantization matrix using the first element associated with the first row and the first column and the element in the first row and the first column of the second quantization matrix; (b) associating a second element in the first one-dimensional matrix, the second element being a difference value between the element in the second row, the first column of the first quantization matrix and the element in the second row, the first column of the second quantization matrix, with the second row, the first column of a two-dimensional matrix as a second element in the predetermined scanning order, and deriving the element in the second row, the first column of the first quantization matrix using the second element associated with the second row, the first column and the element in the second row, the first column of the second quantization matrix; (c) associating a third element in the first one-dimensional matrix, the third element being a difference value between the element in the first row, the second column of the first quantization matrix and the element in the first row, the second column of the second quantization matrix, with the first row, the second column of a two-dimensional matrix as a third element in the predetermined scanning order, and decoding the first quantization matrix by using the third element associated with the first row, the second column and the element in the first row, the second column of the second quantization matrix to derive the element in the first row, the second column of the first quantization matrix; When the decoded first information indicates that a third mode is used, in the decoding step, a plurality of difference values ​​including a difference value between an element in the first row and first column of the first quantization matrix and a predetermined value, and a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order are decoded from the scaling_list_data syntax structure; (a) using a first element in a second one-dimensional matrix obtained from the plurality of difference values, deriving an element in the first row and first column of the first quantization matrix corresponding to a first element in the predetermined scanning order; (b) using a second element in the second one-dimensional matrix, deriving an element in the second row and first column of the first quantization matrix that corresponds to a second element in the predetermined scanning order; (c) using a third element in the second one-dimensional matrix, deriving an element in the first row and second column of the first quantization matrix that corresponds to a third element in the predetermined scanning order; (d) decoding the first quantization matrix using a fourth element in the second one-dimensional matrix to derive an element in the third row and first column of the first quantization matrix that corresponds to the fourth element in the predetermined scanning order; In the decoding step, a quantization matrix having all elements set to 16 may be used as the second quantization matrix; The code amount of the first information indicating that the first mode is used is smaller than the code amount of the first information indicating that the second mode is used.

2. An image decoding method comprising:

14. a quantization step of quantizing transform coefficients of a block of the image using a first quantization matrix; and an encoding step of encoding the first quantization matrix, When a first mode is used to encode the first quantization matrix, in the encoding step, information indicating that the first quantization matrix is ​​the same as a reference second quantization matrix and information for identifying the second quantization matrix are encoded in a scaling_list_data syntax structure; When a second mode is used to code the first quantization matrix, in the coding step, coding information for identifying the second quantization matrix in the scaling_list_data syntax structure; (a) a first element, which is a first element in a predetermined scanning order, and which is an element of a difference value between an element in a first row and a first column of the first quantization matrix and an element in a first row and a first column of the second quantization matrix; (b) a second element in the predetermined scanning order, the second element being a difference value between an element in a second row and a first column of the first quantization matrix and an element in a second row and a first column of the second quantization matrix; (c) encoding information used to derive a first one-dimensional matrix including a third element in the predetermined scanning order, the third element being an element of a difference value between an element in a first row and a second column of the first quantization matrix and an element in a first row and a second column of the second quantization matrix; When a third mode is used to encode the first quantization matrix, the encoding step further comprises: (a) encoding, into the scaling_list_data syntax structure, a difference value between an element in the first row and first column of the first quantization matrix corresponding to the first element in the predetermined scanning order and a predetermined value; (b) encoding, into the scaling_list_data syntax structure, a difference value between an element in the second row and the first column of the first quantization matrix corresponding to a second element in the predetermined scanning order and an element in the first row and the first column of the first quantization matrix; (c) encoding a difference value between an element in the first row and second column of the first quantization matrix corresponding to a third element in the predetermined scanning order and an element in the second row and first column of the first quantization matrix into the scaling_list_data syntax structure; (d) encoding a difference value between an element in the third row and the first column of the first quantization matrix corresponding to a fourth element in the predetermined scanning order and an element in the first row and the second column of the first quantization matrix into the scaling_list_data syntax structure; encoding a difference value between two consecutive elements of the first quantization matrix in the predetermined scanning order; In the encoding step, a quantization matrix in which all elements are set to 16 may be used as the second quantization matrix; The code amount of the information representing that the first mode is used is smaller than the code amount of the information representing that the second mode is used.

13. An image coding method comprising:

15. A program for causing a computer to execute each step of the method according to claim 13 when the program is read and executed by the computer.

16. A program for causing a computer to execute each step of the method according to claim 14 when the program is read and executed by the computer.

Images