Image encoding device, image decoding device, and control method and program thereof

Abstract

To provide an image encoding apparatus and method in which an encoding amount of a quantization matrix used in encoding is suppressed.SOLUTION: The image encoding apparatus includes: a first encoding unit 110 for dividing image data into basic blocks of a preset size, further dividing into sub-blocks of a plurality of preset division patterns, and performing frequency conversion, quantization, and entropy encoding; a holding unit 103 for holding a plurality of quantization matrices corresponding to a sub-block size; a second encoding unit 113 for encoding the quantization matrices; and an integration unit 111 for integrating the encoded data obtained by the first and second encoding units. The holding unit holds elements at positions in a predetermined range from storage positions of elements of DC components as independent values, and holds the same elements in a predetermined number of units at positions beyond the predetermined range for quantization matrices that have the same size as either a horizontal or a vertical size of the base block.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image encoding technique. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter referred to as HEVC) is known as a coding method for compressing and recording moving images. In order to improve coding efficiency, HEVC employs basic blocks larger than conventional macroblocks (16x16 pixels). These large basic blocks are called coding tree units (CTUs) and have a maximum size of 64x64 pixels. CTUs are further divided into subblocks, which are units for prediction and transformation.

[0003] Furthermore, HEVC uses a process called a quantization matrix, which weights coefficients after orthogonal transformation (hereinafter referred to as orthogonal transformation coefficients) according to frequency components. By further reducing data of high frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technology for encoding such a quantization matrix. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-38758 A Summary of the Invention [Problem to be solved by the invention]

[0005] HEVC uses orthogonal transforms, such as 16x16 pixels or 32x32 pixels, which are larger than the conventional 8x8 pixel orthogonal transform, and also uses quantization matrices corresponding to such large orthogonal transforms. In order to prevent the amount of code of the quantization matrix itself from increasing, instead of encoding all of the elements of the 256 quantization matrices corresponding to the 16x16 pixel orthogonal transform, only 64 elements are encoded and expanded by upsampling processing on the decoding side. In addition, the element at the upper left corner of the quantization matrix, which corresponds to the DC component in the frequency domain, is encoded separately. For this reason, in HEVC, when a large orthogonal transform is used, there is a problem that the quantization processing of frequency components other than the DC component cannot be finely controlled.

[0006] The present invention has been made to solve the above-mentioned problems, and aims to provide a technique for encoding an image while suppressing the amount of code of the quantization matrix used in the encoding. [Means for solving the problem]

[0007] In order to solve this problem, for example, an image decoding device of the present invention has the following arrangement. An image decoding device that decodes an image from encoded data, comprising: A decoding means for decoding data indicating a plurality of values ​​corresponding to at least some elements of a quantization matrix used in an inverse quantization process for deriving transform coefficients from the quantized transform coefficients; generation means for deriving the values ​​from the data and generating the quantization matrix based on at least the values; an inverse quantization means for performing the inverse quantization process on a target block using the quantization matrix; having When the target block has a width of 16 pixels and a height of 8 pixels, the generating means Among the plurality of values The first value is derived by subtracting the first value from a predetermined initial value. Associating a first value with a first element in the quantization matrix corresponding to a DC component and with no other elements in the quantization matrix; Among the plurality of values a second value derived by subtracting the first value Associating a second value with a second element adjacent to the right of the first element in the quantization matrix and with no other elements in the quantization matrix; Among the plurality of values a third value derived by subtracting the second value from the A third value is associated with a third element adjacent below the first element in the quantization matrix and a fourth element adjacent below the second element in the quantization matrix, The quantization matrix is ​​generated. Effect of the Invention

[0008] According to the present invention, it is possible to encode an image while suppressing the amount of code of a quantization matrix used in encoding. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image encoding device according to a first embodiment. [Diagram 2] 1 is a block diagram of an image decoding device according to a second embodiment. [Diagram 3] 4 is a flowchart showing an image encoding process in the image encoding device according to the first embodiment. [Figure 4] 10 is a flowchart showing an image decoding process in an image decoding device according to a second embodiment. [Diagram 5] FIG. 1 is a block diagram showing an example of the hardware configuration of a computer that can be used as an image encoding device and a decoding device. [Figure 6] FIG. 2 is a diagram showing an example of a bitstream structure generated according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of sub-block division used in the embodiment. [Figure 8] 5A and 5B are diagrams showing examples of quantization matrices used in the embodiment. [Figure 9] 5A and 5B are diagrams showing examples of quantization matrices used in the embodiment. [Figure 10]4A and 4B are diagrams showing a method of scanning elements of a quantization matrix used in the embodiment. [Figure 11] 5A and 5B are diagrams showing difference value matrices of quantization matrices generated in the embodiment. [Figure 12] 13 is a diagram showing an example of a coding table used for coding a difference value of a quantization matrix. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations in the embodiments shown below are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0011] [First embodiment] 1 is a block diagram showing an image encoding device according to a first embodiment. The image encoding device has a control unit 150 that controls the components described below and is responsible for controlling the entire device. This control unit 150 includes a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM used as a work area for the CPU. Note that an OS (operating system), programs, and parameters stored in the ROM or other non-volatile storage device may be loaded into the RAM, and then the program may be executed under the control of the OS.

[0012] The image encoding device also has an input terminal 101, a block division unit 102, a quantization matrix storage unit 103, a prediction unit 104, a transformation and quantization unit 105, an inverse quantization and inverse transformation unit 106, an image reproduction unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0013] In the above configuration, image data (e.g., 30 frames / second) is supplied from an image generation source (not shown) to an input terminal 101 of this device. Note that the type of image generation source is not important, but it is simply an imaging unit or a video camera, or a hard disk or storage medium that stores image data to be encoded.

[0014] The block division unit 102 divides image data input via an input terminal 101 into a plurality of basic blocks, and supplies the image data in units of basic blocks to a prediction unit 104 at the subsequent stage.

[0015] The quantization matrix storage unit 103 stores a plurality of types of quantization matrices used in the quantization process. A method for generating the quantization matrices will be described later in detail.

[0016] The prediction unit 104 inputs image data in units of basic blocks from the block division unit 102, determines a division method into a plurality of smaller sub-blocks, and performs intra-prediction, which is intra-frame prediction, and inter-prediction, which is inter-frame prediction, for each sub-block to generate predicted image data. Furthermore, the prediction unit 104 calculates and supplies prediction error data, which is the difference between the image data of a sub-block and the predicted image data. The prediction unit 104 also outputs information necessary for prediction, such as information regarding sub-block division, prediction mode, motion vector, and the like, together with the prediction error data. Hereinafter, this information necessary for prediction is referred to as prediction information.

[0017] The transform / quantization unit 105 performs an orthogonal transform (frequency transform) on the prediction error data supplied from the prediction unit 104 on a sub-block basis to obtain transform coefficients. Then, the transform / quantization unit 105 quantizes the transform coefficients of the sub-block of interest using the corresponding quantization matrix stored in the quantization matrix storage unit 103 to obtain quantized coefficients.

[0018] The inverse quantization / inverse transform unit 106 regenerates the transform coefficients by inverse quantizing the quantization coefficients input from the transform / quantization unit 105 using the corresponding quantization matrix stored in the quantization matrix storage unit 103, and further performs an inverse orthogonal transform on the transform coefficients to regenerate the prediction error data.

[0019] The image reproduction unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. The image reproduction unit 107 then generates reproduced image data from the generated predicted image data and the prediction error data from the inverse quantization and inverse transform unit 106, and outputs the reproduced image data to the frame memory 108 again.

[0020] The in-loop filter unit 109 performs in-loop filter processing such as deblocking filtering and sample adaptive offset on the reconstructed image data from the image reconstruction unit 107 stored in the frame memory 108, and stores the image data after filtering back into the frame memory 109.

[0021] The encoding unit 110 encodes the quantization coefficients input from the transform / quantization unit 105 and the prediction information input from the prediction unit 104 to generate and output coded data.

[0022] The quantization matrix encoding unit 113 encodes the quantization matrix input from the quantization matrix storage unit 103 to generate and output encoded data of quantization.

[0023] The integrated coding unit 111 generates header code data by using the coded data of the quantization matrix coded by the quantization matrix coding unit 113. Furthermore, the integrated coding unit 111 forms a bit stream by combining the generated header code data with the coded data of the image input from the coding unit 110, and outputs the bit stream to the outside via the output terminal 112.

[0024] The output destination is not particularly limited, and may be a storage device such as a recording medium, a file server on a network, or the like.

[0025] The image encoding operation in the image encoding device according to the embodiment will be described in more detail below.

[0026] In this embodiment, moving image data is input in frame units, but still image data for one frame may be input. In this embodiment, for ease of explanation, only intra-prediction coding processing is explained, but the present invention is not limited to this and can also be applied to inter-prediction coding processing. Furthermore, in this embodiment, for the sake of explanation, the size of the basic block into which one frame of image is divided in the block division unit 101 is explained as 16 pixels horizontally by 16 pixels vertically (hereinafter simply referred to as 16×16 pixels), but this is merely an example and other sizes may be used.

[0027] Prior to encoding an image, the quantization matrix is ​​encoded.

[0028] First, the quantization matrix storage unit 103 generates and stores a plurality of quantization matrices corresponding to sub-blocks of a plurality of sizes used in encoding. The bold frame indicated by reference numeral 800 in FIG. 8(a) is an example of a quantization matrix corresponding to a size of 8×8 pixels, and the values ​​in the matrix indicate its elements. The element in row 0, column 0 in the upper left corner of the quantization matrix 800 corresponds to a conversion coefficient of a DC component (direct current component), and the other elements correspond to conversion coefficients of AC components (alternating current components). There is no particular limit to the method of determining each element constituting this quantization matrix. For example, a predetermined initial value may be used, or the elements may be set individually. Also, the elements may be generated according to the characteristics of the image.

[0029] Similarly, the quantization matrix storage unit 103 generates quantization matrices according to the sizes of other subblocks. FIG. 8(b) shows a quantization matrix corresponding to the orthogonal transformation of 16×8 pixels in this embodiment. The element in row 0, column 0 corresponding to the DC component is set to "4", and the element in row 0, column 1 adjacent to the DC component is set to "5". The element in row 0, column 1 of the quantization matrix in FIG. 8(b) corresponds to the transform coefficient of the AC component estimated to be the lowest frequency in the shape of the subblock corresponding to the quantization matrix in FIG. 8(b). On the other hand, the remaining elements are configured such that one value is set for two horizontally adjacent elements in order to reduce the amount of code. For example, the same value of "9" is set to the elements in row 0, column 2 and row 0, column 3, and similarly the same value of "10" is set to the elements in row 1, column 2 and row 1, column 3.

[0030] FIG. 8(c) is a quantization matrix corresponding to the orthogonal transformation of 8×16 pixels in this embodiment. The element in row 0, column 0 corresponding to the DC component is set to an individual value such as "4", and the element in row 1, column 0 below is set to "5". The element in row 1, column 0 of the quantization matrix in FIG. 8(c) is an element corresponding to the transform coefficient of the AC component estimated to be the lowest frequency in the shape of the subblock corresponding to the quantization matrix in FIG. 8(c). On the other hand, the remaining elements are configured such that one value is set for two elements adjacent in the vertical direction in order to reduce the amount of code. For example, the same value of "9" is set to the elements in row 2, column 0 and row 3, column 0, and similarly the same value of "10" is set to the elements in row 2, column 1 and row 3, column 1.

[0031] 8(d) is a quantization matrix corresponding to the orthogonal transformation of 16×16 pixels in this embodiment. Individual values ​​are set as elements corresponding to AC components adjacent to the DC component, such as the element at row 0, column 0 corresponding to the DC component being "4", the elements at row 1, column 0 to the right and row 0, column 1 immediately below being "5", and the element at row 1, column 1 to the lower right being "6". Meanwhile, the remaining elements are configured such that one value is set for four horizontally and vertically adjacent 2×2 elements in order to reduce the amount of code.

[0032] As described above, by setting individual values ​​to the elements in the quantization matrix corresponding to the DC component and the elements corresponding to multiple low-frequency components in the vicinity, and setting values ​​on a multiple-element basis to the remaining elements, it becomes easier to reduce the amount of code in the quantization matrix, and it is possible to finely control the quantization of the low-frequency parts to which human vision is sensitive.

[0033] The quantization matrix storage unit 103 stores the quantization matrix thus generated. In this embodiment, the four types of quantization matrices shown in Figs. 8(a) to (d) are stored in a two-dimensional shape, but the elements in the quantization matrix are not limited to this. In addition, it is also possible to store multiple quantization matrices for orthogonal transformation of the same size depending on a prediction method described later, for example, whether intra prediction or inter prediction is used, or whether the encoding target is a luminance block or a chrominance block. In general, in order to realize a quantization process according to the visual characteristics of humans, the elements of the low frequency part corresponding to the upper left part of the quantization matrix are small, and the elements of the high frequency part corresponding to the lower right part are large, as shown in Figs. 8(a) to (d).

[0034] The quantization matrix encoding unit 113 sequentially reads out the quantization matrices stored in a two-dimensional form from the quantization matrix holding unit 106, scans each element, calculates the difference, and generates a one-dimensional matrix. In this embodiment, each quantization matrix shown in Fig. 8(a) to (d) is encoded. Specifically, the quantization matrix encoding unit 113 first uses a scanning method along the arrows shown in Fig. 10(a) for the quantization matrix shown in Fig. 8(a), and calculates the difference between each element and adjacent elements in the scanning order. For example, the quantization matrix of 8 x 8 pixels shown in Fig. 8(a) is scanned by the diagonal scan shown in Fig. 10(a), and after the first element "4" located in the upper left corner, the element "9" located immediately below it is scanned, and the difference of +5 is calculated. In addition, the encoding of the first element of the quantization matrix ("4" in this embodiment) is assumed to calculate the difference from a predetermined initial value (for example, "8"), but of course this is not limited to this, and the difference from an arbitrary value or the value of the first element itself may be used.

[0035] Similarly, for the quantization matrix shown in Fig. 8(b), the quantization matrix encoding unit 113 uses the scanning method along the arrows shown in Fig. 10(b) to calculate the difference between each element and the previous element in the scanning order. For example, the quantization matrix of 16 x 8 pixels shown in Fig. 8(b) is scanned by the scanning method shown in Fig. 10(b), but after the first element "4" located at the top left corner, the element "5" located immediately to the right is scanned and the difference of +1 is encoded. Next, the element "9" located below that is scanned and the difference of +4 is encoded.

[0036] The same is true for Figure 8(c), where the 8x16 pixel quantization matrix shown in Figure 8(c) is scanned by the scanning method shown in Figure 10(c). After the first element "4" in the upper left corner, the element "5" immediately below it is scanned and the difference of +1 is coded. Then the element "9" below that is scanned and the difference of +4 is coded.

[0037] The 16x16 pixel quantization matrix shown in Fig. 8(d) is scanned by the scanning method shown in Fig. 10(d). After the first element 4 located at the top left corner, element 5 located immediately below is scanned and the difference, +1, is encoded. Next, element 5 located to the right of the top left corner is scanned and the difference, 0, is encoded. Then element 6 located below is scanned and the difference, +1, is encoded.

[0038] In this manner, in this embodiment, the quantization matrices of Fig. 8(a) to (d) are generated using the scanning methods of Fig. 10(a) to (d), respectively, to generate one-dimensional difference matrices as shown in Fig. 11(a) to (d). The quantization matrix encoding unit 113 further encodes the difference matrix to generate quantization matrix code data. In this embodiment, the encoding table shown in Fig. 12(a) is used for encoding, but the encoding table is not limited to this, and for example, the encoding table shown in Fig. 12(b) may be used. The quantization matrix code data generated in this manner is supplied to the downstream integrated encoding unit 111.

[0039] 1, the integrated coding unit 111 codes header information necessary for coding the image data, and integrates the coded data of the quantization matrix. Next, the coding of the image data will be described.

[0040] The block division unit 102 divides one frame of image data input from the input terminal 101 into a plurality of basic blocks, and supplies the image data in units of basic blocks to the prediction unit 104. As described above, the size of a basic block in this embodiment is 16×16 pixels.

[0041] The prediction unit 104 executes a prediction process on the image data in units of basic blocks input from the block division unit 102. Specifically, the prediction unit 104 determines a sub-block division method for dividing the basic blocks into smaller sub-blocks, and further determines an intra-prediction mode such as horizontal prediction or vertical prediction for each sub-block. Note that the sub-block division method is determined based on the amount of encoded data of the immediately preceding frame, etc.

[0042] 7(a) to (d) show examples of subblock division patterns. The reference numeral 700 indicating the outer bold frame in the figure represents a basic block, which in this embodiment has a size of 16×16 pixels. The rectangles within the bold frame represent subblocks. FIG. 7(b) shows an example of conventional square subblock division, in which a basic block of 16×16 pixels is divided into four subblocks of 8×8 pixels. Meanwhile, FIGS. 7(c) and (d) show an example of rectangular subblock division, in which a basic block is divided into two vertical subblocks of 8×16 pixels in FIG. 7(c) and two horizontal subblocks of 16×8 pixels in FIG. 7(d).

[0043] In this embodiment, it is decided whether a basic block of 16x16 pixels is not divided (FIG. 7(a)), is divided into two sub-blocks in the vertical direction (FIG. 7(c)), is divided into two sub-blocks in the horizontal direction (FIG. 7(d)), or is divided using a quadtree method (FIG. 7(b)). However, the sub-block division method is not limited to this. Ternary tree division that divides in a ratio of 1:2:1 as shown in FIGS. 7(e) and (f) may also be used.

[0044] Prediction image data is generated from the determined intra prediction mode and the encoded pixels. Then, the prediction unit 104 generates prediction error data, which is the difference between the input image data and the generated predicted image data, for each sub-block, and supplies the prediction error data to the transformation and quantization unit 105. In addition, information such as the sub-block division and the intra prediction mode is supplied to the encoding unit 110 and the image reproduction unit 107 as prediction information.

[0045] The transform / quantization unit 105 performs orthogonal transform / quantization on the input prediction error data to generate quantization coefficients. Specifically, the transform / quantization unit 105 first performs orthogonal transform processing corresponding to the size of the subblock to generate orthogonal transform coefficients. Then, the transform / quantization unit 105 quantizes the orthogonal transform coefficients in units of subblocks using a quantization matrix of a corresponding size stored in the quantization matrix storage unit 103 to generate quantization coefficients. In this embodiment, the quantization matrix of FIG. 8(d) is used for the 16×16 pixel subblock division corresponding to FIG. 7(a). Similarly, the quantization matrix of FIG. 8(c) is used for the 8×16 pixel subblock division of FIG. 7(c), and the quantization matrix of FIG. 8(b) is used for the 16×8 subblock division of FIG. 7(d). Also, the quantization matrix of FIG. 8(a) is used for the subblock division (quadtree division) having a size half that of the basic block both vertically and horizontally as shown in FIG. 7(b). The transform / quantization unit 105 supplies the generated quantized coefficients (transformation coefficients after quantization) to the encoding unit 110 and the inverse quantization / inverse transform unit 106 .

[0046] The inverse quantization and inverse transform unit 106 regenerates transform coefficients by inverse quantizing the input quantization coefficients in sub-block units using the corresponding quantization matrix stored in the quantization matrix holding unit 103. The inverse quantization and inverse transform unit 106 then performs inverse orthogonal transform on the regenerated transform coefficients in sub-block units to regenerate prediction error data, and supplies the regenerated prediction error data to the image reproduction unit 107.

[0047] The image reproduction unit 107 regenerates predicted image data for each sub-block by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. The image reproduction unit 107 regenerates image data for each sub-block by adding the regenerated predicted image data and the prediction error data input from the inverse quantization and inverse transform unit 106, and stores the regenerated image data in the frame memory 108.

[0048] The in-loop filter unit 109 reads out the reconstructed image data from the frame memory in units of subblocks, and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 109 restores the filtered image data in the frame memory .

[0049] The coding unit 110 entropy-encodes the quantization coefficients generated by the transform / quantization unit 105 and the prediction information input from the prediction unit 104 for each subblock, generating coded data. There is no particular limit to the entropy coding method, and Golomb coding, arithmetic coding, Huffman coding, or the like can be used. The generated coded data is supplied to an integrated coding unit 111.

[0050] The integrated encoding unit 111 multiplexes the encoded data of the header and the encoded data input from the encoding unit 110 to form a bit stream. Finally, the bit stream is output from a terminal 112 to the outside.

[0051] FIG. 6(a) shows an example of the data structure of a bit stream output in the first embodiment. The sequence header includes code data corresponding to each quantization matrix in FIG. 8(a) to (d), and is composed of the encoding results of each element. However, the position of encoding is not limited to this, and it is of course possible to have a configuration in which the data is encoded in a picture header section or other header section. In addition, when changing a quantization matrix in one sequence, it is also possible to update the quantization matrix by newly encoding it. In this case, all quantization matrices may be rewritten, or it is also possible to change a part of the quantization matrix by specifying the transform block size of the quantization matrix to be rewritten.

[0052] Hereinafter, a processing procedure for encoding one frame in the image encoding device according to the first embodiment will be described with reference to the flowchart in Fig. 3. Note that each step in the following description is performed under the control of the control unit 150.

[0053] First, prior to encoding an image, in S301, the quantization matrix storage unit 103 generates and stores a quantization matrix. In this embodiment, the quantization matrix storage unit 103 generates and stores four types of quantization matrices shown in Fig. 8(a) to (d).

[0054] In S302, the quantization matrix encoding unit 113 scans the quantization matrix generated and held in S301 to calculate the difference between each element, and generates a one-dimensional difference matrix. In this embodiment, the quantization matrix encoding unit 113 calculates the difference value between two consecutive elements of the quantization matrix of Figs. 8(a) to (d) in the scanning order of Figs. 10(a) to (d), and generates a one-dimensional matrix of difference values ​​shown in Figs. 11(a) to (d). Then, the quantization matrix encoding unit 113 generates binary codes for each difference value in the generated matrix by referring to the encoding table shown in Fig. 12(a) (or Fig. 12(b)), and generates encoded data.

[0055] In S303, the integrated encoding unit 111 generates and outputs header information necessary for encoding image data together with the generated quantization matrix code data.

[0056] In S304, the block division unit 102 divides the input image in units of frames into basic blocks. Then, in S305, the prediction unit 104 inputs image data of one basic block generated in S304 and converts it into sub-blocks. Then, the prediction unit 104 executes a prediction process for each sub-block, and generates prediction information and predicted image data, such as sub-block division information and intra-prediction mode. Then, the prediction unit 104 calculates prediction error data from the image data of each sub-block and the predicted image data.

[0057] In S306, the transform / quantization unit 105 performs an orthogonal transform on the prediction error data of each subblock calculated in S305 to generate a transform coefficient. Furthermore, the transform / quantization unit 105 quantizes the prediction error data of each subblock using the quantization matrix generated and stored in S301 to generate a quantization coefficient. In this embodiment, the quantization matrix of FIG. 8(d) is used for the subblock division corresponding to FIG. 7(a). Similarly, the quantization matrix of FIG. 8(c) is used for the subblock division of FIG. 7(c), and the quantization matrix of FIG. 8(b) is used for the subblock division of FIG. 7(d). Furthermore, the quantization matrix shown in FIG. 8(a) is used for the subblock division divided into quadtrees as in FIG. 7(b).

[0058] In S307, the inverse quantization and inverse transform unit 106 inverse quantizes the quantization coefficients of each sub-block generated in S305 using the quantization matrix generated and stored in S301 to reproduce the transform coefficients. The inverse quantization and inverse transform unit 106 then performs inverse orthogonal transform on the transform coefficients of each sub-block to reproduce prediction error data for each sub-block.

[0059] In S308, the image reproduction unit 107 reproduces predicted image data for each sub-block based on the prediction information generated in S305. The image reproduction unit 107 further adds the prediction error data of the corresponding sub-block generated in S307 to the reproduced predicted image data of each sub-block, thereby reproducing image data for each sub-block.

[0060] In S309, the encoding unit 110 encodes the prediction information generated in S305 and the quantization coefficients generated in S306 for each subblock to generate coded data, and also generates a bitstream including other coded data.

[0061] In S310, the control unit 150 determines whether or not the encoding of all basic blocks in the frame has been completed. If so, the control unit 150 proceeds to S311; if not, the control unit 150 returns the process to S305 to perform encoding of the next basic block.

[0062] In S311, the in-loop filter unit 109 performs in-loop filter processing on the image data reproduced in S308 to generate a filtered image, and then ends the processing.

[0063] In the above process, when encoding the second or subsequent frames of a video, steps S304 to S311 are repeated until the last frame to be encoded is input.

[0064] With the above configuration and operation, the amount of code in the quantization matrix can be reduced by individually encoding the first predetermined number of elements (the DC component element and the adjacent relatively low frequency component element) of the quantization matrix in S302 and encoding the remaining elements in units of multiple elements. Furthermore, the quantization of the low frequency part to which the human visual sense is sensitive can be finely controlled.

[0065] In this embodiment, all four types of quantization matrices in Fig. 8(a) to (d) are coded, but instead of coding the four types of quantization matrices independently, the decoding device may generate the quantization matrix using another quantization matrix. For example, if the 16x8 quantization matrix itself is not coded but is generated using the coded 16x16 quantization matrix, the decoding device will thin out some elements from the quantization matrix in Fig. 8(d) to generate the quantization matrix in Fig. 8(b). Similarly, the decoding device can thin out some elements from the quantization matrix in Fig. 8(d) to generate the quantization matrix in Fig. 8(c). Also, if the method of calculating the element "5" in row 0, column 1 in Fig. 8(b) and the element "5" in row 1, column 0 in Fig. 8(c) is determined between the coding device and the decoding device, the quantization matrix in Fig. 8(b) or Fig. 8(c) can be generated using the quantization matrix in Fig. 8(a). The calculation method in this case is not particularly limited, and may be calculated by adding a specific difference value (+1 in this embodiment) to the element "4" in row 0, column 0, or may be calculated by using interpolation with surrounding elements. Note that in FIG. 8(c), for the elements to the right and below the element corresponding to the DC component of the transform coefficient, +1 is added to the element corresponding to the DC component, and +2 is added to the lower right element.

[0066] Also, along with the elements of the 16×16 quantization matrix, the element “5” in row 0, column 1 of FIG. 8(b) or the element “5” in row 1, column 0 of FIG. 8(c) itself may be encoded.

[0067] This makes it possible to code a bitstream with a further reduced amount of code for the quantization matrix.

[0068] Furthermore, it is also possible to select whether to generate each quantization matrix from another quantization matrix or to code each element, and to code the identifier in the header. For example, information indicating whether to generate each element of the quantization matrix from another quantization matrix or to code it individually may be coded in the header as a quantization matrix coding method information code, and a bitstream shown in Fig. 6(b) may be generated. This makes it possible to selectively generate a bitstream that prioritizes image quality control according to the size of the subblock, or a bitstream with a smaller amount of code for the quantization matrix.

[0069] In this embodiment, the quantization matrices in Fig. 8(a) to Fig. 8(d) corresponding to the subblock division and orthogonal transformation in Fig. 7(a) to Fig. 7(d) are used, but the orthogonal transformation size and quantization matrices used are not limited to this. For example, when ternary tree division that divides at a ratio of 1:2:1 as in Fig. 7(e) and (f) is further used, the quantization matrices in Fig. 9(a) and (b) may be further used.

[0070] As described above, the quantization matrix storage unit 103 in the embodiment stores elements for each position corresponding to each transform coefficient independently in the long side direction of the quantization matrix shown in Figures 8(b) and (c) in which only one of the horizontal and vertical directions of a basic block (16x16 size in the embodiment) has the same size and the other is different, within a predetermined range (a distance of "1" in the embodiment) from the storage position of the element corresponding to the DC component, and stores the same element for a predetermined number of transform coefficients (two in the embodiment) at positions beyond the predetermined range in the long side direction. Meanwhile, in the short side direction of the quantization matrix shown in Figures 8(b) and (c), elements of the quantization matrix are arranged for each transform coefficient.

[0071] For a quantization matrix that is the same size as the basic block in both the horizontal and vertical directions, as shown in Fig. 8, the position of the element corresponding to the DC component and the four adjacent elements to the right, below, and below-right of that position (in this embodiment, within a distance of "1") are stored separately for each corresponding transform coefficient. For the other transform coefficients, one identical element is stored for every four mutually adjacent transform coefficients. For a quantization matrix of 8x8 size as shown in Fig. 8(a), an element is stored for each transform coefficient.

[0072] In this way, the allocation of elements to transform coefficients is adaptively changed according to the size of the quantization matrix, so that the amount of code for the quantization matrix can be reduced, and deterioration of image quality can also be suppressed.

[0073] The "predetermined range" may be defined based on the scanning order as shown in FIG. 10 when encoding elements of a quantization matrix. For a block size of 8x16 or 16x8 transform coefficients, the scanning as shown in FIG. 10(b)(c) is performed to encode / decode elements of a quantization matrix. In this case, in the process of the decoding device described below, an element obtained by one scan (within the range of distance "1") from an element corresponding to a DC component of a transform coefficient is assigned to each transform coefficient, and one element is assigned to every two transform coefficients thereafter. That is, a quantization matrix as shown in FIG. 8(b)(c) is obtained in the process of the decoding device. In addition, in the case of a block size of 16x16 transform coefficients, an element obtained by three scans (within the range of distance "3") from an element corresponding to a DC component of a transform coefficient is assigned to each transform coefficient, and one element is assigned to every four transform coefficients thereafter. That is, a quantization matrix as shown in FIG. 8(d) is obtained. Also, in the case of a block size of 8x8 transform coefficients, elements obtained by all scanning from the element corresponding to the DC component of the transform coefficient are assigned to each transform coefficient. That is, a quantization matrix as shown in FIG. 8(a) is obtained in the process of the decoding device. In this way, the allocation of elements to transform coefficients may be adaptively changed according to the size of the quantization matrix based on the scanning of the quantization matrix. In this case, it is not necessary to take into special consideration the positional relationship between elements.

[0074] [Second embodiment] In the second embodiment, an image decoding device that decodes encoded data generated by the image encoding device of the first embodiment will be described. Fig. 2 is a block diagram showing the configuration of the image decoding device.

[0075] The image decoding device includes a control unit 250 that controls the components described below and controls the entire device. The control unit 250 includes a CPU, a ROM that stores programs executed by the CPU and various parameters, and a RAM used as a work area for the CPU. Note that an OS (operating system), programs, and parameters stored in the ROM or other non-volatile storage device may be loaded into the RAM, and then the program may be executed under the control of the OS.

[0076] The image decoding device also has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization and inverse transform unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0077] The configuration and operation of the image decoding device in the configuration of the image decoding device will be described below. Note that the image decoding device is configured to input the bit stream generated by the image encoding device described above in units of frames, but it may also be configured to input a still image bit stream for one frame. In addition, in this second embodiment, for ease of explanation, only intra prediction decoding processing will be described, but the present invention is not limited to this and can also be applied to inter prediction decoding processing.

[0078] The separate decoding unit 202 receives an encoded bitstream via an input terminal 201, separates the bitstream into encoded data related to information on decoding processing and coefficients, and decodes the encoded data present in the header of the bitstream. The separate decoding unit 202 of this embodiment separates encoded data of a quantization matrix from the encoded bitstream and supplies the encoded data to a quantization matrix decoding unit 209. The separate decoding unit 202 also separates encoded data of image data from the encoded bitstream and supplies the encoded data to a decoding unit 203. In short, the separate decoding unit 202 performs the reverse operation of the integrated encoding unit 111 in FIG. 1.

[0079] The quantization matrix decoding unit 209 decodes the coded data of the quantization matrix supplied from the separation decoding unit 202, and reproduces and stores the quantization matrix.

[0080] The decoding unit 203 decodes the coded data of the image data input from the separate decoding unit 202, and reproduces the quantization coefficients and prediction information.

[0081] Similar to the inverse quantization and inverse transform unit 106 in FIG. 1, the inverse quantization and inverse transform unit 204 performs inverse quantization on the quantized coefficients using the reproduced quantization matrix to obtain transform coefficients, and then performs inverse orthogonal transform to reproduce the prediction error data.

[0082] The image reproduction unit 205 generates predicted image data based on the input prediction information by appropriately referring to the frame memory 206. Then, the image reproduction unit 205 adds the prediction error data reproduced by the inverse quantization and inverse transform unit 204 to this predicted image data, thereby reproducing image data and storing it in the frame memory 206.

[0083] 1, the in-loop filter unit 207 performs in-loop filtering such as deblocking filtering on the reconstructed image data stored in the frame memory 207, and stores the filtered image data back in the frame memory. Then, the output terminal 208 outputs the reconstructed image data to an external device (e.g., a display device).

[0084] Next, the operation of each component in the image decoding device in FIG. 2 will be described in more detail.

[0085] In the second embodiment, the bit stream generated in the first embodiment is input in units of frames, but a still image bit stream for one frame may be input. In addition, in the present embodiment, for ease of explanation, only intra prediction decoding processing is explained, but the present embodiment is not limited to this and can also be applied to inter prediction decoding processing.

[0086] A bitstream for one frame input via an input terminal 201 is supplied to a separate decoding unit 202. The separate decoding unit 202 separates the bitstream into information related to the decoding process and coded data related to coefficients, and decodes the coded data present in the header of the bitstream. More specifically, the separate decoding unit 202 reproduces coded data of a quantization matrix. The separate decoding unit 202 of this embodiment first extracts coded data of a quantization matrix from the sequence header of the bitstream shown in FIG. 6(a), and supplies it to a quantization matrix decoding unit 209. The separate decoding unit 202 then reproduces coded data in sub-block units of a basic block of picture data, and supplies it to a decoding unit 203.

[0087] The quantization matrix decoding unit 209 first decodes the code data of the input quantization matrix to reproduce a one-dimensional difference matrix. In this embodiment, as in the first embodiment, the decoding is performed using the coding table shown in FIG. 12(a) (or FIG. 12(b)). However, the coding table is not limited to this, and other coding tables may be used as long as they are the same as those in the first embodiment. Furthermore, the quantization matrix decoding unit 209 inversely scans the reproduced one-dimensional difference matrix to reproduce a two-dimensional quantization matrix. Here, the operation is reverse to the scanning of the quantization matrix encoding unit 113 in the first embodiment. That is, the quantization matrix decoding unit 209 in this embodiment reproduces and holds the quantization matrices shown in FIGS. 8(a) to 8(d) using the scanning methods shown in FIGS. 10(a) to 8(d) for the difference matrices shown in FIGS. 11(a) to 11(d), respectively.

[0088] A specific quantization matrix decoding process in this embodiment will be described by taking the decoding process of the 16×8 quantization matrix in Fig. 8(b) as an example. The quantization matrix decoding unit 209 reproduces the quantization matrix shown in Fig. 8(b) by using the scanning method shown in Fig. 10(b) for the difference matrix shown in Fig. 11(b). The quantization matrix decoding unit 209 calculates the first element "4" from the initial value "8" and the first difference value "-4" in Fig. 11(b), determines the value "4" of the element at the position of row 0, column 0, which is the first scanning position in Fig. 10(b), and places the element "4" in the part of row 0, column 0 in Fig. 8(b). Next, the quantization matrix decoding unit 209 calculates the second element "5" from the element "4" in row 0, column 0 of FIG. 8(b) and the second difference value "1" in FIG. 11(b), and places "5" in the part of row 0, column 1, which is the second scanning position of FIG. 10(b). Similarly, the third element "9" is calculated from the element "5" in row 0, column 1 of FIG. 8(b) and the third difference value "4" in FIG. 10(b), and places "9" in the parts of row 1, column 0 and row 1, column 1, which are the third scanning position of FIG. 9(b). The same process is repeated to reproduce the quantization matrix shown in FIG. 8(b).

[0089] In this way, by assigning individual values ​​to multiple elements corresponding to the low-frequency components of the quantization matrix, and assigning values ​​in multiple-element units to the remaining elements, it is possible to reduce the amount of code while decoded a bitstream in which the quantization of the low-frequency parts to which human vision is sensitive is finely controlled.

[0090] 2, the decoding unit 203 decodes the coded data of each sub-block in the basic block to regenerate the quantization coefficients and prediction information. The decoding unit 203 supplies the regenerated quantization coefficients to the inverse quantization and inverse transform unit 204 and the regenerated prediction information to the image reproduction unit 205.

[0091] The inverse quantization and inverse transform unit 204 performs inverse quantization on the input quantization coefficients in subblock units using the quantization matrix reproduced and stored in the quantization matrix decoding unit 209, and generates orthogonal transform coefficients. The inverse quantization and inverse transform unit 204 further performs inverse orthogonal transform on the generated orthogonal transform coefficients in subblock units to reproduce prediction error data, and supplies the data to the image reproduction unit 205. At this time, the quantization matrix to be used is determined according to the size of the subblock to be decoded, and in this embodiment, the quantization matrix of FIG. 8(d) is used for the subblock division corresponding to FIG. 7(a). Similarly, the quantization matrix of FIG. 8(c) is used for the subblock division of FIG. 7(c), and the quantization matrix of FIG. 8(b) is used for the subblock division of FIG. 7(d). Also, the quantization matrix shown in FIG. 8(a) is used for the subblock division of the quadtree of FIG. 7(b).

[0092] The image reproduction unit 205 appropriately refers to the frame memory 206 for each sub-block based on the prediction information input from the decoding unit 203, and reproduces predicted image data. The image reproduction unit 205 then adds the prediction error data supplied from the inverse quantization and inverse transform unit 204 to this predicted image data, thereby reproducing image data of the sub-block, and stores the image data in the frame memory 206. The image data stored in this frame memory 206 becomes a prediction reference candidate when decoding other sub-blocks.

[0093] 1, the in-loop filter unit 207 reads out reconstructed image data from the frame memory 206 in units of subblocks, and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 207 stores the filtered image back in the frame memory 206.

[0094] The reproduced image stored in the frame memory 206 is finally outputted from a terminal 208 to the outside.

[0095] Hereinafter, a processing procedure of one frame decoding processing in the image decoding device according to the second embodiment will be described with reference to the flowchart in Fig. 4. Note that each step in the following description is executed under the control of the control unit 250.

[0096] First, in S401, the demultiplexing / decoding unit 202 demultiplexes the bit stream into information related to the decoding process and coded data related to coefficients, and decodes the coded data of the header portion. More specifically, the demultiplexing / decoding unit 202 reproduces the coded data of the quantization matrix.

[0097] In S402, the quantization matrix decoding unit 209 decodes the encoded data of the quantization matrix reproduced in S401 to reproduce a one-dimensional difference matrix as shown in FIG. 10. Furthermore, the quantization matrix decoding unit 209 inversely scans the reproduced one-dimensional difference matrix to reproduce a two-dimensional quantization matrix. That is, the quantization matrix decoding unit 209 reproduces and holds the quantization matrices shown in FIG. 8(a)-(d) by using the scanning methods shown in FIG. 10(a)-(d) for the difference matrices shown in FIG. 11(a)-(d), respectively. Note that the scanning methods shown in FIG. 10(a)-(d) and the allocation of the elements of the quantization matrix to the transform coefficients are known to the decoding side device. That is, each element of the quantization matrix is ​​assigned to the transform coefficient according to the distance from the element corresponding to the DC component of the transform coefficient. That is, in the long side direction of the quantization matrix as shown in Figures 8(b) and (c), where only one of the horizontal and vertical directions of a basic block (16x16 size in this embodiment) has the same size and the other is different, within a predetermined range (a distance of "1" in this embodiment) from the storage position of an element corresponding to a DC component, the quantization matrix decoding unit 209 assigns one element to each position corresponding to each transform coefficient, and at positions beyond the predetermined range in the long side direction, assigns the same element to a predetermined number of transform coefficients (two in this embodiment). On the other hand, in the short side direction of the quantization matrix as shown in Figures 8(b) and (c), an element of the quantization matrix is ​​assigned to each transform coefficient.

[0098] For a quantization matrix that is the same size as the basic block in both the horizontal and vertical directions, the quantization matrix decoding unit 209 assigns the position of an element corresponding to a DC component and four adjacent elements (within a distance of "1" in this embodiment) to the right, bottom, and bottom right of that position individually to each corresponding transform coefficient, as shown in Fig. 8. For other transform coefficients, the same element is assigned to every four mutually adjacent transform coefficients. For a quantization matrix of 8x8 size as shown in Fig. 8(a), an element is held for each transform coefficient.

[0099] In this way, the allocation of elements to transform coefficients is adaptively changed according to the size of the quantization matrix, so that the amount of code for the quantization matrix can be reduced, and deterioration of image quality can also be suppressed.

[0100] The "predetermined range" may be defined based on the scanning order as shown in FIG. 10 when decoding elements of a quantization matrix. For a block size of 8x16 or 16x8 transform coefficients, the scanning as shown in FIG. 10(b)(c) is performed to decode elements of a quantization matrix. At this time, the quantization matrix decoding unit 209 assigns elements obtained by one scan (within the distance "1") from elements corresponding to the DC components of the transform coefficients to each transform coefficient, and thereafter assigns one element for every two transform coefficients. That is, a quantization matrix as shown in FIG. 8(b)(c) is obtained. Also, for a block size of 16x16 transform coefficients, the quantization matrix decoding unit 209 assigns elements obtained by three scans (within the distance "3") from elements corresponding to the DC components of the transform coefficients to each transform coefficient, and thereafter assigns one element for every four transform coefficients. That is, a quantization matrix as shown in FIG. 8(d) is obtained. Also, in the case of a block size of 8x8 transform coefficients, the quantization matrix decoding unit 209 extracts elements obtained by all scanning from elements corresponding to DC components of the transform coefficients for each transform coefficient. That is, a quantization matrix as shown in FIG. 8(a) is obtained. In this way, the allocation of elements to transform coefficients may be adaptively changed for the size of the quantization matrix based on the scanning of the quantization matrix. In this case, it is not necessary to take into special consideration the positional relationship between elements.

[0101] In S403, the decoding unit 203 decodes the coded data separated in S401, and reproduces the quantized coefficients and prediction information for each sub-block.

[0102] In S404, the inverse quantization and inverse transform unit 204 performs inverse quantization on the quantized coefficients for each sub-block using the quantization matrix reconstructed in S402 to obtain transform coefficients. The inverse quantization and inverse transform unit 204 further performs inverse orthogonal transform on the transform coefficients to reconstruct prediction error data for each sub-block.

[0103] In S405, the image reproduction unit 205 reproduces the prediction information and predicted image data for each sub-block generated in S403. Then, the image reproduction unit 205 reproduces image data by adding the prediction error data generated in S404 to the reproduced predicted image data for each sub-block.

[0104] In S406, the control unit 250 determines whether or not the decoding of all basic blocks in the frame has been completed. If the decoding has been completed, the process proceeds to S407. If not, the process returns to S403 to decode the next basic block.

[0105] In S407, the in-loop filter unit 207 performs in-loop filter processing on the image data reproduced in S405, generates a filtered image, and ends the processing.

[0106] When decoding the second and subsequent frames of a video, the processes from S403 to S407 are repeated until the end of the encoded bit stream or until a stop instruction is received from the user via an operation unit (not shown).

[0107] With the above configuration and operation, it is possible to decode a bitstream with a reduced amount of code of the quantization matrix generated in the first embodiment, in which only the first several elements of the quantization matrix are coded individually and the remaining elements are coded in units of several elements.

[0108] In the second embodiment, all four types of quantization matrices in Fig. 8(a) to (d) are decoded from the bit stream. Alternatively, other quantization matrices may be generated from one quantization matrix decoded from the bit stream. For example, the 16x8 quantization matrix itself may not be decoded from the bit stream, but may be generated using a 16x16 quantization matrix decoded from the bit stream. In this case, some elements are thinned out from the quantization matrix in Fig. 8(d) to generate the quantization matrix in Fig. 8(b). Similarly, some elements can be thinned out from the quantization matrix in Fig. 8(d) to generate the quantization matrix in Fig. 8(c). In addition, if a calculation method for the element "5" in row 0, column 1 of Fig. 8(b) or the element "5" in row 1, column 0 of Fig. 8(c), which is the above-mentioned "predetermined range", is determined, the quantization matrix in Fig. 8(b) or Fig. 8(c) can be generated using the quantization matrix in Fig. 8(a). In this case, the calculation method is not particularly limited, and may be calculated by adding a specific difference value (+1 in this embodiment) to the element "4" in row 0, column 0, or may be calculated by using interpolation with surrounding elements. In addition, the element "5" in row 0, column 1 of FIG. 8(b) or the element "5" in row 1, column 0 of FIG. 8(c) itself may be decoded from the bitstream along with the elements of the 16×16 quantization matrix. This makes it possible to decode a bitstream with a further reduced amount of code of the quantization matrix.

[0109] Furthermore, an identifier for selecting whether to generate each quantization matrix from another quantization matrix or to decode each element may be decoded from the header. For example, information indicating whether to generate each element of the quantization matrix from another quantization matrix or to decode it individually may be decoded as a quantization matrix encoding method information code from the header part of the bit stream shown in Fig. 6(b). This makes it possible to decode a bit stream in which a priority is given to image quality control according to the size of the sub-block or the amount of code of the quantization matrix is ​​reduced.

[0110] In this embodiment, the quantization matrices in Fig. 8(a) to Fig. 8(d) corresponding to the subblock division and orthogonal transformation in Fig. 7(a) to Fig. 7(d) are used, but the orthogonal transformation size and quantization matrices used are not limited to this. For example, when ternary tree division that divides at a ratio of 1:2:1 as in Fig. 7(e) and (f) is further used, the quantization matrices in Fig. 9(a) and (b) may be further used.

[0111] [Third embodiment] In the above embodiment, each processing unit shown in Fig. 1 and Fig. 2 is configured as hardware. However, the processing performed by each processing unit shown in these figures may be configured as a computer program.

[0112] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer applicable to the image display device according to each of the above embodiments.

[0113] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and ROM 503, and executes the processes described above as being performed by the image processing device according to each of the above embodiments. That is, the CPU 501 functions as each of the processing units shown in FIGS.

[0114] The RAM 502 has an area for temporarily storing computer programs and data loaded from an external storage device 506, data acquired from the outside via an I / F (interface) 507, etc. Furthermore, the RAM 502 has a work area used when the CPU 501 executes various processes. That is, the RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as appropriate.

[0115] The ROM 503 stores setting data for the computer, a boot program, etc. The operation unit 504 is composed of a keyboard, a mouse, etc., and can be operated by a user of the computer to input various instructions to the CPU 501. The display unit 505 displays the results of processing by the CPU 501. The display unit 505 is composed of, for example, a liquid crystal display.

[0116] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores an OS (operating system) and computer programs for causing the CPU 501 to realize the functions of each unit shown in Figures 1 and 2. Furthermore, the external storage device 506 may store each image data to be processed.

[0117] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501, and become the subject of processing by the CPU 501. A network such as a LAN or the Internet, and other devices such as a projector or display device can be connected to the I / F 507, and the computer can obtain and transmit various information via this I / F 507. Reference numeral 508 denotes a bus that connects the above-mentioned components.

[0118] In the above configuration, when the power supply of the present device is turned on, the CPU 501 executes a boot program in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and starts the OS. As a result, the present device becomes capable of communication via the interface 507 and functions as an information processing device. Then, under the control of the OS, the CPU 501 loads an application (corresponding to FIG. 3) relating to image coding from the external storage device 506 into the RAM 502 and executes it, so that the CPU 501 functions as the various processing units shown in FIG. 1, and the present device functions as an image coding device. On the other hand, when the CPU 501 loads an application (corresponding to FIG. 4) relating to image decoding from the external storage device 506 into the RAM 502 and executes it, the CPU 501 functions as the various processing units shown in FIG. 2, and the present device functions as an image decoding device.

[0119] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Explanation of symbols]

[0120] 101...input terminal, 102...block division unit, 103...quantization matrix storage unit, 104...prediction unit, 105...transformation and quantization unit, 106...inverse quantization and inverse transformation unit, 107...image reproduction unit, 108...frame memory, 109...in-loop filter unit, 110...encoding unit, 111...integrated encoding unit, 112...output terminal, 113...quantization matrix encoding unit

Claims

1. An image decoding device that decodes an image from encoded data, comprising: A decoding means for decoding data indicating a plurality of values ​​corresponding to at least some elements of a quantization matrix used in an inverse quantization process for deriving transform coefficients from the quantized transform coefficients; generation means for deriving the values ​​from the data and generating the quantization matrix based on at least the values; an inverse quantization means for performing the inverse quantization process on a target block using the quantization matrix; having When the target block has a width of 16 pixels and a height of 8 pixels, the generating means a first value among the plurality of values, the first value being derived by a difference from a predetermined initial value, is associated with a first element corresponding to a DC component in the quantization matrix, and is not associated with other elements in the quantization matrix; a second value among the plurality of values, the second value being derived by a difference from the first value, is associated with a second element adjacent to the right of the first element in the quantization matrix, and is not associated with any other element in the quantization matrix; a third value among the plurality of values, the third value being derived by a difference between the third value and the second value, is associated with a third element adjacent below the first element in the quantization matrix, and a fourth element adjacent below the second element in the quantization matrix, Generate the quantization matrix 2. An image decoding device comprising:

2. The third value of the plurality of values ​​is not associated with any element other than the third and fourth elements in the quantization matrix.

2. The image decoding device according to claim 1.

3. The data indicating the plurality of values ​​indicates a difference between the plurality of values.

2. The image decoding device according to claim 1.

4. A selection means for selecting a quantization matrix to be used in the inverse quantization process depending on whether intra prediction or inter prediction is used.

2. The image decoding device according to claim 1, further comprising:

5. A selection means for selecting a quantization matrix to be used in the inverse quantization process depending on whether the block to be decoded is a luminance block or a chrominance block.

2. The image decoding device according to claim 1, further comprising:

6. The number of the plurality of values ​​is one more than half the number of elements constituting the quantization matrix.

2. The image decoding device according to claim 1.

7. An image encoding device that encodes an image, comprising: quantization means for performing a quantization process on a target block using a quantization matrix; an encoding means for encoding data indicating a plurality of values ​​corresponding to at least some elements of the quantization matrix; having When the width of the target block is 16 pixels and the height is 8 pixels, the encoding means data indicating a difference between a first value, which is associated with a first element corresponding to a DC component in the quantization matrix and is not associated with other elements in the quantization matrix, and a predetermined initial value; data indicating a difference between the first value and a second value that is associated with a second element adjacent to the right of the first element in the quantization matrix and is not associated with any other element in the quantization matrix; data indicating a difference between the second value and a third value associated with a third element adjacent below the first element in the quantization matrix and a fourth element adjacent below the second element in the quantization matrix; and encoding data indicating the plurality of values ​​by at least encoding the 1. An image encoding device comprising:

8. The third value is not associated with any element in the quantization matrix other than the third and fourth elements.

8. The image encoding device according to claim 7.

9. The data indicating the plurality of values ​​indicates a difference between the plurality of values.

8. The image encoding device according to claim 7.

10. A selection means for selecting a quantization matrix to be used in the quantization process depending on whether intra prediction or inter prediction is used.

8. The image encoding device according to claim 7, further comprising:

11. A selection means for selecting a quantization matrix to be used in the quantization process depending on whether the target block is a luminance block or a chrominance block.

8. The image encoding device according to claim 7, further comprising:

12. The number of the plurality of values ​​is one more than half the number of elements constituting the quantization matrix.

8. The image encoding device according to claim 7.

13. 1. An image decoding method for decoding an image from encoded data, comprising the steps of: a decoding step of decoding data indicating a plurality of values ​​corresponding to at least some elements of a quantization matrix used in an inverse quantization process for deriving transform coefficients from the quantized transform coefficients; a generating step of deriving the values ​​from the data and generating the quantization matrix based on at least the values; an inverse quantization step of performing the inverse quantization process on the target block using the quantization matrix; having If the target block has a width of 16 pixels and a height of 8 pixels, the generating step a first value among the plurality of values, the first value being derived by a difference from a predetermined initial value, is associated with a first element corresponding to a DC component in the quantization matrix, and is not associated with other elements in the quantization matrix; a second value among the plurality of values, the second value being derived by a difference from the first value, is associated with a second element adjacent to the right of the first element in the quantization matrix, and is not associated with any other element in the quantization matrix; a third value among the plurality of values, the third value being derived by a difference between the third value and the second value, is associated with a third element adjacent below the first element in the quantization matrix, and a fourth element adjacent below the second element in the quantization matrix, Generate the quantization matrix 2. An image decoding method comprising:

14. The third value of the plurality of values ​​is not associated with any element other than the third and fourth elements in the quantization matrix. The image decoding method according to claim 13 .

15. The data indicating the plurality of values ​​indicates a difference between the plurality of values. The image decoding method according to claim 13 .

16. a selection step of selecting a quantization matrix to be used in the inverse quantization process depending on whether intra-prediction or inter-prediction is used; 14. The image decoding method according to claim 13, further comprising:

17. a selection step of selecting a quantization matrix to be used in the inverse quantization process depending on whether the block to be decoded is a luminance block or a chrominance block; 14. The image decoding method according to claim 13, further comprising:

18. The number of the plurality of values ​​is one more than half the number of elements constituting the quantization matrix. The image decoding method according to claim 13 .

19. 1. An image encoding method for encoding an image, comprising the steps of: a quantization step of performing a quantization process on a target block using a quantization matrix; an encoding step of encoding data indicating a plurality of values ​​corresponding to at least some elements of the quantization matrix; having If the current block has a width of 16 pixels and a height of 8 pixels, then the encoding step data indicating a difference between a first value, which is associated with a first element corresponding to a DC component in the quantization matrix and is not associated with other elements in the quantization matrix, and a predetermined initial value; data indicating a difference between the first value and a second value that is associated with a second element adjacent to the right of the first element in the quantization matrix and is not associated with any other element in the quantization matrix; data indicating a difference between the second value and a third value associated with a third element adjacent below the first element in the quantization matrix and a fourth element adjacent below the second element in the quantization matrix; and encoding data indicating the plurality of values ​​by at least encoding the 13. An image coding method comprising:

20. The third value is not associated with any element in the quantization matrix other than the third and fourth elements.

20. The image coding method according to claim 19.

21. The data indicating the plurality of values ​​indicates a difference between the plurality of values.

20. The image coding method according to claim 19.

22. a selection step of selecting a quantization matrix to be used in the quantization process depending on whether intra-prediction or inter-prediction is used; 20. The method of claim 19, further comprising:

23. a selection step of selecting a quantization matrix to be used in the quantization process depending on whether the target block is a luminance block or a chrominance block; 20. The method of claim 19, further comprising:

24. The number of the plurality of values ​​is one more than half the number of elements constituting the quantization matrix.

20. The image coding method according to claim 19.

25. A program for causing a computer to function as each of the means of the image decoding device according to any one of claims 1 to 6.

26. A program causing a computer to function as each of the means of the image encoding device according to any one of claims 7 to 12.

Images