Image decoding device, method, and program

Abstract

To prevent a deterioration in subjective image quality, while increasing the efficiency in encoding the residual coefficients of two color difference components.SOLUTION: An image encoding device encodes image data composed of one luminance component and two color difference components, and has: quantization means that performs orthogonal transformation on components of blocks obtained by dividing an image to be encoded, and quantizes conversion factors of the components obtained through the orthogonal transformation; and encoding means that encodes residual coefficients obtained by the quantization means. When integrating and encoding the conversion factors of the two color difference components obtained through the orthogonal transformation, the encoding means performs quantization on an orthogonal transformation factor obtained through the integration by using a predetermined quantization matrix, and encodes a residual coefficient obtained.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image decoding technique. [Background technology]

[0002] The High Efficiency Video Coding (HEVC) coding method (hereafter referred to as HEVC) is known as a coding method for compressing and recording moving images. In order to improve coding efficiency, HEVC employs basic blocks larger than conventional macroblocks (16x16 pixels). These large basic blocks are called coding tree units (CTUs) and have a maximum size of 64x64 pixels. CTUs are further divided into subblocks, which are units for prediction and transformation. Furthermore, HEVC uses a process called a quantization matrix, which weights coefficients after orthogonal transformation (hereinafter referred to as orthogonal transformation coefficients) according to frequency components. By further reducing data of high frequency components, degradation of which is less noticeable to human vision, it is possible to improve compression efficiency while maintaining image quality. Patent Document 1 discloses a technology for encoding such a quantization matrix.

[0003] In recent years, activities have been started to internationally standardize a more efficient coding method as a successor to HEVC. JVET (Joint Video Experts Team) was established between ISO / IEC and ITU-T, and standardization is underway as the Versatile Video Coding (VVC) coding method (hereinafter referred to as VVC). To improve coding efficiency, a new coding method (hereinafter referred to as chrominance residual coefficient integrated coding) is being considered for VVC, which collectively codes the residual coefficients after quantization of the two chrominance components (Cb component and Cr component). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-38758 A Summary of the Invention [Problem to be solved by the invention]

[0005] In VVC, as in HEVC, the introduction of a quantization matrix is ​​being considered. However, the quantization matrix in HEVC is premised on quantization and inverse quantization processing for each of the luma (Y) and chrominance (Cb, Cr) components, and is not compatible with chrominance residual coefficient integrated coding, which is a new coding method. For this reason, when chrominance residual coefficient integrated coding is used, there is a problem that quantization control according to frequency components cannot be performed, and subjective image quality cannot be improved. Therefore, the present invention has been made to solve the above-mentioned problem, and aims to improve subjective image quality by enabling quantization processing using a quantization matrix that also supports chrominance residual coefficient integrated coding. [Means for solving the problem]

[0006] In order to solve this problem, for example, an image coding device of the present invention has the following arrangement. An image decoding device that decodes encoded data of an image, comprising: A decoding means for decoding data corresponding to quantized transform coefficients, which are quantized transform coefficients, from the encoded data; a derivation means for deriving a prediction error from the decoded data by at least performing inverse quantization using a quantization matrix; having The prediction error of the chrominance component can be derived using the first mode or the second mode; the image comprises a plurality of color difference components; the first mode is a mode in which a sign of a first quantized transform coefficient associated with a first chrominance component among the plurality of chrominance components is omitted in the coded data, and the first chrominance component and a second chrominance component among the plurality of chrominance components are integrated; In the first mode, the deriving means derives a first prediction error for the first chrominance component using a quantized transform coefficient associated with a chrominance component other than the first chrominance component; In the first mode, the derivation means dequantizes the quantized transform coefficient used to derive the first prediction error of the first color difference component using a quantization matrix corresponding to the color difference component different from the first color difference component; the second mode is a mode in which signs of the first quantized transform coefficients are not omitted in the encoded data; the quantization matrix used for inverse quantization of the quantized transform coefficients used to derive the first prediction error when the first mode is used is the same as the quantization matrix used for inverse quantization of the quantized transform coefficients associated with a chrominance component other than the first chrominance component when the second mode is used; The prediction error is for a block partitioned by one or more partitioning methods, including ternary tree partitioning. It is characterized by: Effect of the Invention

[0007] According to the present invention, it is possible to suppress deterioration of subjective image quality while improving the coding efficiency of residual coefficients of two color difference components. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an image encoding device according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configuration of an image decoding device according to the first embodiment. [Diagram 3] 5 is a flowchart showing an image encoding process in the image encoding device according to the first embodiment. [Figure 4] 5 is a flowchart showing an image decoding process in the image decoding device according to the first embodiment. [Diagram 5] FIG. 11 is a diagram showing the configuration of computer hardware applied in the second embodiment. [Figure 6] FIG. 4 is a diagram showing an example of the data structure of a bitstream generated in the embodiment. [Figure 7] FIG. 4 is a diagram showing an example of sub-block division in the embodiment. [Figure 8] 5A and 5B are diagrams showing examples of quantization matrices used in the embodiment. [Figure 9] 4A and 4B are diagrams showing a method of scanning elements of a quantization matrix used in the embodiment. [Figure 10] 6A and 6B are diagrams showing difference value matrices of quantization matrices generated in the embodiment. [Figure 11] 13 is a diagram showing an example of a coding table used for coding a difference value of a quantization matrix. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] 1 is a block diagram of an image encoding device according to this embodiment. The image encoding device has a control unit 150 which controls the entire device. The control unit 150 has a CPU, a ROM for storing programs executed by the CPU, and a RAM used as a work area for the CPU. The image encoding device also has an input terminal 101, a block division unit 102, a quantization matrix holding unit 103, a prediction unit 104, a transform / quantization unit 105, an inverse quantization / inverse transform unit 106, an image reproduction unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0011] An input terminal 101 inputs image data to be encoded, generated in an image data generation source, in frame units. The image data generation source may be of any type, such as an imaging device, a file server or storage medium that stores image data to be encoded, etc. An output terminal 112 outputs the encoded data to a destination device, which may also be a storage medium, a file server, etc., and is not particularly limited.

[0012] The block division unit 102 divides an input frame image into a plurality of basic blocks, and sequentially outputs each of the blocks as a basic block to the downstream prediction unit 104 .

[0013] The quantization matrix storage unit 103 generates a plurality of quantization matrices prior to encoding and stores them in an internal memory (not shown). There is no particular limitation on the method of generating the quantization matrices, but the quantization matrix may be input by the user, may be calculated from the characteristics of the input image, or may be a pre-specified initial value. The quantization matrix storage unit 103 in this embodiment generates and stores a two-dimensional quantization matrix 801 corresponding to the orthogonal transformation of 8×8 pixels shown in FIG. 8(a) and two-dimensional quantization matrices 801 and 802 corresponding to the orthogonal transformation of 4×4 pixels shown in FIG. 8(b) and (c). The quantization matrix 801 is for the luminance component, and the quantization matrices 801 and 802 are for two chrominance components.

[0014] The prediction unit 104 determines subblock division for basic block unit image data, and performs intra-frame prediction or inter-frame prediction on a subblock unit basis to generate predicted image data. Furthermore, the prediction unit 104 calculates and outputs a prediction error on a pixel basis from the input image data and predicted image data. The prediction unit 104 also outputs information necessary for prediction, such as subblock division, prediction mode, motion vector, and the like, together with the prediction error. Hereinafter, the information necessary for prediction is referred to as prediction information.

[0015] The transform / quantization unit 105 performs orthogonal transform on the prediction errors input from the prediction unit 104 in units of sub-blocks to obtain orthogonal transform coefficients (orthogonally transformed residual coefficients). Furthermore, the transform / quantization unit 105 quantizes the orthogonal transform coefficients using the quantization matrix stored in the quantization matrix storage unit 103 to obtain quantized residual coefficients (quantized orthogonal transform coefficients).

[0016] The inverse quantization and inverse transform unit 106 receives the residual coefficients from the transform and quantization unit 105, inverse quantizes them using the quantization matrix stored in the quantization matrix storage unit 103, and reproduces the orthogonal transform coefficients. The inverse quantization and inverse transform unit 106 further performs inverse orthogonal transform to reproduce the prediction error.

[0017] The image reproduction unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image reproduction unit 107 generates reproduced image data by adding the prediction error from the inverse quantization and inverse transform unit 106 to this predicted image data, and stores the reproduced image data in the frame memory 108.

[0018] An in-loop filter unit 109 performs in-loop filtering such as deblocking filtering and sample adaptive offset on the reconstructed image stored in the frame memory 108, and stores the image data after the filtering processing in the frame memory 108 again.

[0019] The encoding unit 110 encodes the residual coefficients output from the transform / quantization unit 105 and the prediction information output from the prediction unit 104 to generate coded data, and outputs the coded data to the integrated encoding unit 111 .

[0020] The quantization matrix encoding unit 113 encodes the quantization matrix stored in the quantization matrix storage unit 103 to generate quantization matrix code data, and outputs the quantization matrix code data to the integrated encoding unit 111 .

[0021] The integrated coding unit 111 generates header code data including the quantization matrix code data from the quantization matrix coding unit 113. Then, the integrated coding unit 111 forms a bit stream by appending the code data output from the coding unit 110 to the header code data. Then, the integrated coding unit 111 outputs the formed bit stream via an output terminal 112.

[0022] Here, the image encoding operation in the image encoding device will be described in more detail below. In this embodiment, moving image data in 4:2:0 color format is input from the input terminal 101 in frame units, but still image data for one frame may be input. In this embodiment, for ease of explanation, only intra-prediction encoding processing will be described, but the present invention is not limited to this and can also be applied to inter-prediction encoding processing. Furthermore, for the sake of explanation, this embodiment will be described assuming that the block division unit 101 divides image data input from the input terminal into basic blocks of 8×8 pixels. That is, the basic block of 8×8 pixels includes luminance (Y) component pixels of 8×8 pixels and chrominance (Cb and Cr) component pixels of 4×4 pixels. Note that this is for ease of understanding and is not limited to the above numerical values ​​(sizes).

[0023] Prior to encoding an image, a quantization matrix is ​​generated and encoded.

[0024] The quantization matrix storage unit 103 first generates and stores a quantization matrix. Specifically, the quantization matrix is ​​generated according to the size of the subblock to be coded and the type of prediction method. In this embodiment, the quantization matrix is ​​generated corresponding to a basic block of 8×8 pixels that is not divided into subblocks, as shown in FIG. 7(a). That is, the quantization matrix storage unit 103 generates an 8×8 quantization matrix for the luminance (Y) component and a 4×4 quantization matrix for the chrominance (Cb and Cr) components. However, the quantization matrix to be generated is not limited to this, and a quantization matrix corresponding to the shape of the subblock, such as 4×8 or 8×4, may be generated. There is no particular limit to the method of determining each element constituting the quantization matrix. For example, a predetermined initial value may be used, or the elements may be set individually. The quantization matrix may also be generated according to the characteristics of the image.

[0025] The quantization matrix holding unit 103 holds the quantization matrix thus generated in an internal memory (not shown). FIG. 8(a) shows a quantization matrix 800 for the Y component. FIG. 8(b) shows a quantization matrix 801 for the Cb component, and FIG. 8(c) shows a quantization matrix 802 for the Cr component. For ease of explanation, the quantization matrix is ​​assumed to be configured for 64 pixels of 8×8 and 16 pixels of 4×4, and each square in a thick frame represents each element constituting the quantization matrix. In this embodiment, the three types of quantization matrices shown in FIG. 8(a) to (c) are held in a two-dimensional shape, but the elements in the quantization matrix are not limited to this. In addition, it is also possible to hold multiple quantization matrices for the same color component depending on the size of the subblock or the type of prediction method described later, for example, whether intra prediction or inter prediction is used. Generally, a quantization matrix realizes quantization processing that corresponds to the characteristics of human vision, so that the elements in the low-frequency portion corresponding to the upper left part of the quantization matrix are small and the elements in the high-frequency portion corresponding to the lower right part are large, as shown in Figures 8(a) to (c).

[0026] The quantization matrix encoding unit 113 sequentially reads out the two-dimensional quantization matrices stored in the quantization matrix storage unit 106, calculates the difference between the previous and next elements obtained by scanning each element, and arranges them in a one-dimensional matrix. In this embodiment, the scanning method shown in FIG. 9(a) or (b) is used for each quantization matrix shown in FIG. 8(a) to (c) according to the size, and calculates the difference between each element and the previous element in the scanning order. For example, the quantization matrix of 4×4 pixels shown in FIG. 8(c) is scanned by the scanning method shown in FIG. 9(b), and after the first element 8 located in the upper left, the element 14 located immediately below it is scanned, and the difference of +6 is calculated. In addition, the encoding of the first element of the quantization matrix (8 in this embodiment) is calculated by calculating the difference from a predetermined initial value (for example, 8), but of course it is not limited to this, and the difference from an arbitrary value or the value of the first element itself may be used. In short, it is sufficient that the initial value is the same as that of the decoding device.

[0027] In this way, the quantization matrix encoding unit 113 generates a difference matrix for the quantization matrix 800 in FIG. 8(a) using the scanning method in FIG. 9(a). Similarly, the quantization matrix encoding unit 113 generates a difference matrix for the quantization matrices 801 and 802 in FIG. 8(b) and (c) using the scanning method in FIG. 9(b). FIG. 10(a) shows a difference matrix 1000 obtained from the quantization matrix 800 in FIG. 8(a). FIG. 10(b) shows a difference matrix 1001 obtained from the quantization matrix 801 in FIG. 8(b). And FIG. 10(c) shows a difference matrix 1002 obtained from the quantization matrix 802.

[0028] The quantization matrix encoding unit 113 further encodes each difference matrix generated as described above to generate quantization matrix code data. The quantization matrix encoding unit 113 of this embodiment encodes each element of the difference matrix by assigning a code word to each element using the encoding table shown in Fig. 11(a) to generate quantization matrix code data. Note that the encoding table is not limited to this, and for example, the encoding table shown in Fig. 11(b) may be used. In this way, the quantization matrix encoding unit 113 outputs the generated quantization matrix code data to the integrated encoding unit 111 at the subsequent stage.

[0029] Returning to FIG. 1, the integrated encoding unit 111 integrates the encoded data of the quantization matrix with header information necessary for encoding image data.

[0030] Next, the image data is encoded. One frame of image data input from an input terminal 101 is supplied to a block division unit 102. The block division unit 102 divides the input image data of one frame into a plurality of basic blocks, and outputs the image data in units of basic blocks to a prediction unit 104. In this embodiment, image data in units of basic blocks of 8×8 pixels is supplied to the prediction unit 104.

[0031] The prediction unit 104 performs prediction processing on the image data in units of basic blocks input from the block division unit 102. Specifically, the prediction unit 104 determines sub-block division for dividing the basic block into smaller sub-blocks, and further determines an intra-prediction mode such as horizontal prediction or vertical prediction for each sub-block.

[0032] The subblock division method will be described with reference to FIG. 7. The bold frames of blocks 700 to 705 in FIG. 7(a) to (f) are the same size as the basic block, 8×8 pixels. Each square in the bold frame represents a subblock. FIG. 7(b) shows an example of conventional square subblock division, in which a basic block 701 of 8×8 pixels is divided into four subblocks of 4×4 pixels. Meanwhile, FIG. 7(c) to (f) show an example of rectangular subblock division. FIG. 7(c) shows that a basic block 702 is divided into two subblocks (vertically extending) of 4×8 pixels. FIG. 7(d) shows that a basic block 703 is divided into two subblocks (horizontally extending) of 8×4 pixels. In the case of basic blocks 704 and 705 in FIG. 7(e) and (f), although the division directions are different, they are divided into three rectangular subblocks at a ratio of 1:2:1. In this way, not only square but also rectangular subblocks are used for coding processing.

[0033] In this embodiment, only the basic block of 8×8 pixels is used as shown in FIG. 7(a), which is not divided into subblocks, but the subblock division method is not limited to this. Quadtree division as shown in FIG. 7(b), ternary tree division as shown in FIG. 7(e) or (f), or binary tree division as shown in FIG. 7(c) or FIG. 7(d) may be used. When a subblock division other than that shown in FIG. 7(a) is used, a quantization matrix corresponding to the subblock to be used is generated by the quantization matrix storage unit 103. The generated quantization matrix is ​​coded by the quantization matrix coding unit 113.

[0034] The prediction unit 104 generates predicted image data from the determined prediction mode and the encoded area stored in the frame memory 108, calculates a prediction error in pixel units from the input image data and the predicted image data, and outputs the prediction error to the transformation and quantization unit 105. The prediction unit 104 also outputs information such as sub-block division and prediction mode as prediction information to the encoding unit 110 and image reproduction unit 107.

[0035] The transform and quantization unit 105 performs orthogonal transform and quantization on the prediction error input from the prediction unit 104 to generate residual coefficients. In parallel with this, the transform and quantization unit 105 determines whether or not to use chrominance residual coefficient integrated coding, which collectively codes the residual coefficients of the Cb and Cr components.

[0036] Here, the chrominance residual coefficient integrated coding used in this embodiment will be described. The chrominance residual coefficient integrated coding is a coding method for reducing the amount of generated code and improving coding efficiency by coding only one of the residual coefficients when the correlation between the Cb and Cr residual coefficients is high, and omitting the coding of the other residual coefficient. On the decoding side, only one of the coded chrominance residual coefficients is decoded, and the other chrominance residual coefficient is reconstructed by calculating it using the decoded chrominance residual coefficient. In this embodiment, as an example, when a negative correlation is found between the Cb residual coefficient and the Cr residual coefficient, the chrominance residual coefficient integrated coding is used, and only the component Cb is coded in the downstream coding unit 110, and the coding of the Cr residual coefficient is omitted.

[0037] Returning to FIG. 1, the transform / quantization unit 105 first performs an orthogonal transform process corresponding to the size of the subblock on the prediction error to generate orthogonal transform coefficients (orthogonally transformed residual coefficients). Next, the transform / quantization unit 105 determines whether or not to use chrominance residual coefficient integrated coding based on the correlation between the generated orthogonal transform coefficients of the components Cb and Cr. That is, the transform / quantization unit 105 determines whether or not the orthogonal transform coefficient of the component Cb multiplied by (-1) is close to the orthogonal transform coefficient of the component Cr, and if it is determined that it is close, determines to use chrominance residual coefficient integrated coding. In addition, the transform / quantization unit 105 generates information (1 bit may be sufficient) indicating whether or not to use chrominance residual coefficient integrated coding as chrominance integrated information. Then, the transform / quantization unit 105 quantizes the orthogonal transform coefficients using a quantization matrix stored in the quantization matrix storage unit 103 according to the color component, and generates residual coefficients (quantized orthogonal transform coefficients). In this embodiment, the quantization matrix of FIG. 8(a) is used for the orthogonal transform coefficient of the Y component, the quantization matrix of FIG. 8(b) is used for the orthogonal transform coefficient of the Cb component, and the quantization matrix of FIG. 8(c) is used for the orthogonal transform coefficient of the Cr component. However, the quantization matrix to be used is not limited to this. In addition, when it is determined to use the chrominance residual coefficient integration coding, the orthogonal transform coefficient of the Cb component is quantized using the quantization matrix of FIG. 8(b), and the quantization process of the orthogonal transform coefficient of the Cr component and the output to the subsequent stage are omitted. This makes it possible to reduce the amount of calculation for the quantization process of the Cr component that is not used in the subsequent process. The generated residual coefficient and chrominance integration information are output to the coding unit 110 and the inverse quantization and inverse transform unit 106.

[0038] The inverse quantization and inverse transform unit 106 inverse quantizes the residual coefficients (quantized orthogonal transform coefficients) input from the transform and quantization unit 105 using the quantization matrix stored in the quantization matrix storage unit 103, thereby reproducing the orthogonal transform coefficients (more precisely, residual coefficients that have not been quantized but have been orthogonally transformed). The inverse quantization and inverse transform unit 106 further performs inverse orthogonal transform on the reproduced orthogonal transform coefficients to reproduce the prediction error. For the inverse quantization process, a quantization matrix corresponding to the color components of the subblock to be coded is used, as in the transform and quantization unit 105. Specifically, the inverse quantization and inverse transform unit 106 performs inverse quantization using the same quantization matrix as that used by the transform and quantization unit 105. That is, the quantization matrix in FIG. 8(a) is used for the residual coefficients of the Y component, the quantization matrix in FIG. 8(b) is used for the residual coefficients of the Cb component, and the quantization matrix in FIG. 8(c) is used for the residual coefficients of the Cr component.

[0039] Furthermore, when the chrominance integration information indicates that chrominance residual coefficient integration coding is used in the sub-block, the inverse quantization and inverse transform unit 106 inverse quantizes the residual coefficients of the Cb component using the quantization matrix of FIG. 8(b) to reproduce the orthogonal transform coefficients of the Cb component. The inverse quantization and inverse transform unit 106 multiplies the reproduced orthogonal transform coefficients of the Cb component by "-1" to reproduce the orthogonal transform coefficients of the Cr component. The prediction errors reproduced by performing inverse orthogonal transform on the orthogonal transform coefficients reproduced in this manner are output to the image reproduction unit 107.

[0040] The image reproduction unit 107 appropriately refers to the frame memory 108 and reproduces a predicted image based on the prediction information input from the prediction unit 104. The image reproduction unit 107 then generates reproduced image data based on the reproduced predicted image and the prediction error reproduced by the inverse quantization and inverse transform unit 106, and stores the reproduced image data in the frame memory 108.

[0041] The in-loop filter unit 109 reads out the reproduced image data from the frame memory 108, and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 109 stores the filtered image data back in the frame memory 108.

[0042] The encoding unit 110 performs entropy coding on the residual coefficients and color difference integration information for each subblock generated by the transform / quantization unit 105, and the prediction information input from the prediction unit 104, to generate coded data. When the color difference integration information indicates that the color difference residual coefficient integration coding is not to be used in the subblock, the encoding unit 110 encodes the residual coefficients of all color components Y, Cb, and Cr. On the other hand, when the color difference integration information indicates that the color difference residual coefficient integration coding is to be used in the subblock, the encoding unit 110 encodes the residual coefficients of the color components Y and Cb, but does not encode the residual coefficient of the Cr component. The method of entropy coding is not particularly specified, but Golomb coding, arithmetic coding, Huffman coding, and the like can be used. The encoding unit 110 outputs the generated coded data to the integration coding unit 111.

[0043] The integrated coding unit 111 multiplexes the coded data of the header described above and the coded data input from the coding unit 110 to form a bit stream. Then, the integrated coding unit 111 outputs the formed bit stream from an output terminal 112 to the outside (a recording medium such as a network).

[0044] FIG. 6(a) shows an example of the data structure of a bit stream output in this embodiment. The sequence header includes the coded data of the quantization matrix, and is composed of the coding results of each element. However, the coding position is not limited to this, and it is of course possible to code the quantization matrix in a picture header section or other header section. In addition, when changing the quantization matrix in one sequence, it is also possible to update the quantization matrix by newly coding it. In this case, all the quantization matrices may be rewritten, or it is also possible to change a part of the quantization matrix by specifying the color components of the quantization matrix corresponding to the quantization matrix to be rewritten.

[0045] FIG. 3 is a flowchart showing the encoding process for one frame performed by the control unit 150 in the image encoding device according to the embodiment.

[0046] First, prior to encoding an image, in S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold a two-dimensional quantization matrix. The quantization matrix holding unit 103 of this embodiment corresponds to a block of 8×8 pixels, and generates and holds quantization matrices 801 and 802 corresponding to the color components of the Y component, Cb component, and Cr component shown in Fig. 8(a) to (c).

[0047] In S302, the control unit 150 controls the quantization matrix encoding unit 113 to scan the quantization matrix generated and held in S301, calculate the difference between each element, and generate a difference matrix. In this embodiment, the quantization matrices 801 to 803 shown in Figs. 8(a) to (c) are scanned using the scanning method of Fig. 9(a) or (b), and the difference matrices shown in Figs. 10(a) to (c) are generated. Then, the control unit 150 controls the quantization matrix encoding unit 113 to encode the generated difference matrix, and generate quantization matrix code data.

[0048] At S303, the control unit 150 controls the integrated coding unit 111 to code and output header information required for coding image data together with the generated quantization matrix code data. At S304, the control unit 150 controls the block division unit 102 to divide the input image in units of frames into basic blocks. At S305, the control unit 150 controls the prediction unit 104 to execute a prediction process on the image data in units of basic blocks generated at S304, and generate prediction information such as sub-block division information and a prediction mode, and predicted image data. Furthermore, the control unit 150 controls the prediction unit 104 to calculate a prediction error from the input image data and predicted image data.

[0049] In S306, the control unit 150 controls the transform / quantization unit 105 to perform orthogonal transform on the prediction error calculated in S305, and generate transform coefficients. Furthermore, the control unit 150 controls the transform / quantization unit 105 to perform quantization using the quantization matrix generated and held in S301, and generate residual coefficients. In parallel with this, the transform / quantization unit 105 determines whether or not to use chrominance residual coefficient integrated coding, which collectively codes the residual coefficients of Cb and Cr. In this embodiment, the quantization matrix in FIG. 8(a) is used for the orthogonal transform coefficients of the Y component, the quantization matrix in FIG. 8(b) is used for the orthogonal transform coefficients of the Cb component, and the quantization matrix in FIG. 8(c) is used for the orthogonal transform coefficients of the Cr component. In addition, when the chrominance residual coefficient integrated coding is used, the orthogonal transform coefficients of the Cb component are quantized using the quantization matrix in FIG. 8(b), and the quantization process of the orthogonal transform coefficients of the Cr component is omitted.

[0050] As a specific example of the judgment process for determining whether or not to use chrominance residual coefficient integrated coding, a specific judgment process for determining whether or not the orthogonal transform coefficient of component Cb multiplied by (-1) is close to the orthogonal transform coefficient of component Cr is as follows.

[0051] The orthogonal transform coefficients of the components Cb and Cr are each 4 × 4. Here, the transform coefficients in the scan order of Fig. 9(b) are represented as Cb(i) and Cr(i) (i = 0, 1, ... 15). Cr(i) is said to be close to the value obtained by multiplying Cb(i) by "-1" if the relationship between the two for i = 0, 1, 2, ... can be regarded as the following relationship. Cb(i) ≒ -Cr(i) That is, the sum of the color differences Cb and Cr is equal to or smaller than the threshold value as shown in the following equation. |Cb(i)+Cr(i)|≦threshold Here, |x| represents the absolute value of the real number x.

[0052] In this embodiment, the number of coefficients of the color difference components is 4 × 4. Therefore, if the following equation is satisfied using a predetermined threshold value Th, it can be determined that the orthogonal transform coefficient of component Cb multiplied by (-1) is close to the orthogonal transform coefficient of component Cr. Σ|Cb(i)+Cr(i)| <Th (Here, Σ denotes the summation of i=0,1,…15) Instead of finding the absolute value, the sum of the squares of the sums of the two color differences on the left side of the following equation may be used for the determination. Σ{Cb(i)+Cr(i)} 2 <Th In the above embodiment, the chrominance residual coefficient joint coding is used when a negative correlation is observed between the Cb residual coefficient and the Cr residual coefficient. However, this is not limited to the above. Cb(i) ≒ aCr(i) (a is an integer) It is also possible to generalize the correlation as follows: In this case, the determination process for determining whether or not to use the chrominance residual coefficient integrated coding is Σ{|Cb(i)|-|Cr(i) / a|} <Th or, Σ{|Cb(i)|-|Cr(i) / a|} 2 <Th When this condition is satisfied, chrominance residual coefficient joint coding is applied.

[0053] The information "a" indicating the correlation may be encoded.

[0054] In S307, the control unit 150 controls the inverse quantization and inverse transform unit 106 to inverse quantize the residual coefficients generated in S306 using the quantization matrix generated and held in S301, thereby reproducing the orthogonal transform coefficients. In this step, the same quantization matrix as that used in S306 is used to perform the inverse quantization process. When chrominance residual coefficient integrated coding is used, the residual coefficients of the Cb component are inverse quantized using the quantization matrix of FIG. 8(b) to reproduce the orthogonal transform coefficients of the Cb component. Furthermore, the orthogonal transform coefficients of the reproduced Cb component are multiplied by "-1" to reproduce them as the orthogonal transform coefficients of the Cr component. Then, the reproduced orthogonal transform coefficients are inversely orthogonally transformed to reproduce the prediction error.

[0055] In S308, the control unit 150 controls the image reproduction unit 107 to reproduce a predicted image based on the prediction information generated in S305, reproduce image data from the reproduced predicted image and the prediction error generated in S307, and store the image data in the frame memory 108.

[0056] In S309, the control unit 150 controls the encoding unit 110 to encode the prediction information generated in S305 and the residual coefficients and chrominance integration information generated in S306, and generate coded data. Here, when the chrominance integration information indicates that chrominance residual coefficient integration coding is not used in the sub-block, the encoding unit 110 encodes the residual coefficients of all color components Y, Cb, and Cr. On the other hand, when the chrominance integration information indicates that chrominance residual coefficient integration coding is used in the sub-block, the encoding unit 110 encodes the residual coefficients of the color components Y and Cb, and omits encoding the residual coefficient of the Cr component. Note that, instead of encoding the residual coefficient of the Cb component, the average value of the residual coefficient of the Cb component and the value obtained by inverting the sign of the residual coefficient of the Cr component may be encoded as the encoded data of the color components when chrominance residual coefficient integration coding is used. In this case, it is possible to suppress deterioration of the color reproduction of the image after decoding. Furthermore, the encoding unit 110 outputs the generated coded data to the integrated encoding unit 111. The integrated encoding unit 111 positions the coded data from the encoding unit 110 so that it follows the header generated earlier, and outputs the coded data.

[0057] In S310, the control unit 150 determines whether or not the coding of all basic blocks in the frame of interest has been completed. If the control unit 150 determines that the coding has been completed, the process proceeds to S311, and if the control unit 150 determines that there are basic blocks remaining that have not been coded, the process returns to S304 and coding continues for the next basic block.

[0058] In S311, the control unit 150 controls the in-loop filter unit 109 to perform in-loop filter processing on the image data reproduced in S308, generate a filtered image, and then terminate the processing.

[0059] With the above configuration and operation, particularly in S306, quantization processing using a quantization matrix is ​​performed on sub-blocks using chrominance residual coefficient joint coding, thereby controlling quantization for each frequency component and improving subjective image quality.

[0060] In this embodiment, the quantization matrix of FIG. 8(b) is used when chrominance residual coefficient joint coding is used, but the quantization matrix used is not limited to this. For example, the effect is the same even if the quantization / inverse quantization process is performed using the quantization matrix for the Cr component of FIG. 8(c). In addition, it is also possible to generate a new quantization matrix using the existing quantization matrix of FIG. 8(b) or FIG. 8(c) and use it for the quantization / inverse quantization process.

[0061] Furthermore, a configuration may be adopted in which a separate quantization matrix is ​​coded for the chrominance residual coefficient integrated coding. In addition to the existing quantization matrices shown in Figs. 8(a) to (c), a quantization matrix 803 for the chrominance residual coefficient integrated coding may be used for coding as shown in Fig. 8(d) and used for quantization processing of sub-blocks in which the chrominance residual coefficient integrated coding is used. This makes it possible to realize unique quantization control for sub-blocks in which the chrominance residual coefficient integrated coding is used.

[0062] Furthermore, in this embodiment, the quantization matrix for the sub-block using the chrominance residual coefficient integrated coding is uniquely determined, but an identifier may be introduced to make the quantization matrix selectable. For example, in Fig. 6(b), chrominance matrix information (2 bits) for specifying the quantization matrix is ​​newly introduced to make the quantization matrix for the sub-block using the chrominance residual coefficient integrated coding selective.

[0063] For example, when the chrominance matrix information is 0, the quantization matrix for the Cb component in FIG. 8(b) is used for the sub-block that used the chrominance residual coefficient integrated coding. When the chrominance matrix information indicates 1, the quantization matrix for the Cr component in FIG. 8(c) is used for the sub-block that used the chrominance residual coefficient integrated coding. On the other hand, when the chrominance matrix information indicates 2, the quantization matrix individually encoded in FIG. 8(d) is used for the sub-block that used the chrominance residual coefficient integrated coding. This makes it possible to selectively realize quantization matrix code amount reduction and unique quantization control for the sub-block that used the chrominance residual coefficient integrated coding.

[0064] To explain the above in more detail, when the chrominance integration information of a certain sub-block (in the embodiment, the sub-block is defined as a basic block) of a certain picture indicates that chrominance residual coefficient integration coding is not used, the transform coefficient of the Cb component of the sub-block is quantized using the quantization matrix 801 shown in Figure 8(b), and the transform coefficient of the Cr component is quantized using the quantization matrix 802 in Figure 8(c).

[0065] On the other hand, when the chrominance integration information of a certain subblock of a certain picture indicates that the chrominance residual coefficient integration coding is performed, which of the subblocks of the components Cb and Cr is coded depends on the chrominance matrix information stored in the header. For example, when the chrominance matrix information is 0, it indicates that the transform coefficients of the subblock of the component Cb of the components Cb and Cr are quantized and coded using the quantization matrix 801 in FIG. 8(b), and the quantization and coding of the component Cr are omitted. When the chrominance matrix information is 1, it indicates that the transform coefficients of the subblock of the component Cr of the components Cb and Cr are quantized and coded using the quantization matrix 802 in FIG. 8(c), and the quantization and coding of the component Cb are omitted (the component Cb may be coded and the component Cr may be omitted). When the chrominance matrix information is 2, it indicates that the transform coefficients of the sub-block of component Cb (or component Cr) of components Cb and Cr are quantized and encoded using quantization matrix 803 in FIG. 8(d), and that the quantization and encoding of component Cr (or component Cb) is omitted.

[0066] 8(d), the header stores encoded data of the quantization matrix 803. In other words, the quantization matrix storage unit 103 generates and stores the quantization matrix 803, and the quantization matrix encoding unit 113 also encodes the quantization matrix 803.

[0067] 2 is a block diagram of an image decoding device that decodes the encoded image data generated by the image encoding device according to the embodiment. The configuration and operation related to the decoding process will be described below with reference to the diagram.

[0068] The image decoding device has a control unit 250 which controls the entire device. The control unit 250 has a CPU, a ROM which stores programs executed by the CPU, and a RAM which is used as a work area for the CPU. The image decoding device also has an input terminal 201, a separation decoding unit 202, a decoding unit 203, an inverse quantization and inverse transform unit 204, an image reproduction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0069] An input terminal 201 is used to input an encoded bit stream. The input source may be, for example, a storage medium that stores an encoded stream, but may also be input from a network, and the type of input source is not important.

[0070] The separate decoding unit 202 separates the bitstream into information related to the decoding process and coded data related to coefficients, and decodes the coded data present in the header part of the bitstream. The separate decoding unit 202 of this embodiment separates quantization matrix coded data and outputs it to the quantization matrix decoding unit 209. The separate decoding unit 202 also outputs image coded data to the decoding unit 203. That is, the separate decoding unit 202 performs the reverse operation of the integrated coding unit 111 in FIG. 1.

[0071] The quantization matrix decoding unit 209 reproduces and stores the quantization matrix by decoding the quantization matrix code data supplied from the separation decoding unit 202 .

[0072] The decoding unit 203 decodes the coded image data output from the separation decoding unit 202, and reproduces the residual coefficients, the integrated color difference information, and the prediction information.

[0073] Similar to the inverse quantization and inverse transform unit 106 in FIG. 1 , the inverse quantization and inverse transform unit 204 performs inverse quantization on the residual coefficients using the reconstructed quantization matrix to obtain the inverse quantized coefficients, and then performs inverse orthogonal transform to reconstruct the prediction error.

[0074] The image reproduction unit 205 generates predicted image data based on the input prediction information by appropriately referring to the frame memory 206. The image reproduction unit 205 then generates reconstructed image data from this predicted image data and the prediction error reconstructed by the inverse quantization and inverse transform unit 204, and stores it again in the frame memory 206.

[0075] Similar to in-loop filter unit 109 in FIG. 1, in-loop filter unit 207 performs in-loop filtering such as deblocking filtering on the reproduced image data stored in frame memory 206, and stores the image data after filtering back in the frame memory.

[0076] The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. The output destination is generally a display device, but may be other devices.

[0077] The image decoding operation of the image decoding device of the above embodiment will now be described in more detail. In this embodiment, an encoded bit stream is input in units of frames.

[0078] In FIG. 2, a bit stream for one frame input from an input terminal 201 is supplied to a separate decoding unit 202. The separate decoding unit 202 separates the bit stream into information related to the decoding process and coded data related to coefficients, and decodes the coded data present in the header of the bit stream. The separate decoding unit 202 then supplies the quantization matrix coded data contained in the header to a quantization matrix decoding unit 209, and supplies coded data of image data to the decoding unit 202. Specifically, the separate decoding unit 202 first extracts quantization matrix coded data from the sequence header of the bit stream shown in FIG. 6(a), and outputs it to the quantization matrix decoding unit 209. In this embodiment, quantization matrix coded data corresponding to the quantization matrices shown in FIGS. 8(a) to 8(c) is extracted and output. Next, coded data for each basic block of picture data is extracted, and output to the decoding unit 203.

[0079] The quantization matrix decoding unit 209 first decodes the input quantization matrix code data to reproduce the one-dimensional difference matrices 1000 to 1002 shown in Figs. 10(a) to (c). The quantization matrix decoding unit 209 then decodes each quantization matrix using the coding table shown in Fig. 11(a) as in the encoding side. In the embodiment, the coding table in Fig. 11(a) is used, but the coding table in Fig. 11(b) may be used. In short, it is sufficient to use the same one as the encoding side. The quantization matrix decoding unit 209 inversely scans the reproduced one-dimensional difference matrices 1000 to 1002, reproduces the two-dimensional quantization matrix, and holds it. Here, the operation is reverse to that of the quantization matrix encoding unit 113 on the encoding side. That is, the quantization matrix encoding unit 113 of this embodiment decodes the difference matrices 1000 to 1002 shown in Figs. 10(a) to (c) using the scanning method shown in Fig. 9(a) or (b) according to the size of each matrix. As a result, the quantization matrix encoding unit 113 can reproduce and store the three types of quantization matrices 800 to 802 shown in FIGS. 8(a) to 8(c).

[0080] The decoding unit 203 decodes the encoded data supplied from the separation decoding unit 202, reproduces the chrominance integration information and the prediction information, and further reproduces the residual coefficients according to the reproduced chrominance integration information. First, the decoding unit 203 reproduces the prediction information and obtains the prediction mode used in the sub-block. Next, the decoding unit 203 reproduces the chrominance integration information and determines whether or not the sub-block is subjected to chrominance residual coefficient integrated coding. If the chrominance integration information indicates that the sub-block does not use chrominance residual coefficient integrated coding, the decoding unit 203 decodes and reproduces the residual coefficients of all the color components Y, Cb, and Cr. On the other hand, if the chrominance integration information indicates that the sub-block uses chrominance residual coefficient integrated coding, the decoding unit 203 decodes the residual coefficients of the Y and Cb color components (since there is no encoded data of the Cr color component, the decoding process is not performed). The decoding unit 203 outputs the reconstructed color difference integrated information and residual coefficients to the inverse quantization and inverse transform unit 204 , and outputs the reconstructed prediction information to the image reconstruction unit 205 .

[0081] The inverse quantization and inverse transform unit 204 performs inverse quantization on the input residual coefficients using the quantization matrix reproduced by the quantization matrix decoding unit 209 to generate orthogonal transform coefficients, and further performs inverse orthogonal transform to reproduce prediction errors. The inverse quantization and inverse transform unit 204 performs inverse quantization using a quantization matrix corresponding to the color components of the subblock to be decoded, similar to the inverse quantization and inverse transform unit 106 on the coding side. That is, the quantization matrix of FIG. 8(a) is used for the residual coefficients of the Y component, the quantization matrix of FIG. 8(b) is used for the residual coefficients of the Cb component, and the quantization matrix of FIG. 8(c) is used for the residual coefficients of the Cr component. However, when the chrominance integration information indicates that chrominance residual coefficient integration coding is used in the subblock, the decoding unit 203 does not supply the residual coefficients of the orthogonal transform coefficients of the Cr component. Therefore, the inverse quantization and inverse transform unit 204 performs inverse quantization on the residual coefficients of the Cb component using the quantization matrix of FIG. 8(b), and then reproduces the orthogonal transform coefficients of the Cr component. That is, the quantization matrix is ​​used to perform inverse quantization before reconstructing the Cr component, thereby improving the efficiency of the process. Specifically, the inverse quantization and inverse transform unit 204 reproduces the orthogonal transform coefficients of the reconstructed Cb component by (-1) as the orthogonal transform coefficients of the Cr component, and supplies the result to the image reconstruction unit 205. The prediction error reproduced by performing inverse orthogonal transform on the orthogonal transform coefficients thus reproduced is output to the image reconstruction unit 205. However, the quantization matrix used is not limited to this, and may be the same as the quantization matrix used in the transform and quantization unit 105 and the inverse quantization and inverse transform unit 106 on the encoding side.

[0082] In the above, a case where a negative correlation is observed has been described as an example, but the present invention is not limited to this. Cb(i) ≒ aCr(i) (a is an integer) If the correlation is found and chrominance residual coefficient integrated coding is performed, the orthogonal transform coefficient of the Cb component is multiplied by (a) to be reproduced as the orthogonal transform coefficient of the Cr component. The information "a" indicating the correlation may be decoded from the bit stream, or may be stored in advance as a rule on the coding side and the decoding side.

[0083] The image reproduction unit 205 appropriately refers to the frame memory 206 based on the prediction information input from the decoding unit 203, and reproduces a predicted image. The image reproduction unit 205 of this embodiment uses intra prediction such as horizontal prediction and vertical prediction, similar to the prediction unit 104 on the encoding side. The specific prediction process is similar to that of the prediction unit 104 on the encoding side, and therefore a description thereof will be omitted. The image reproduction unit 205 reproduces image data from this predicted image and the prediction error input from the inverse quantization and inverse transform unit 204, and re-stores the image data in the frame memory 206. The stored image data is used as a reference when making predictions.

[0084] The in-loop filter unit 207, like the in-loop filter unit 109 on the encoding side, reads out a reconstructed image from the frame memory 206 and performs in-loop filtering such as deblocking filtering. Then, the in-loop filter unit 207 restores the filtered image to the frame memory 206.

[0085] The reproduced image stored in the frame memory 206 is finally output from an output terminal 208 to an external device (typically a display device).

[0086] FIG. 4 is a flowchart showing the decoding process of the control unit 250 in the image decoding device according to the embodiment.

[0087] First, in S401, the control unit 250 controls the separation decoding unit 202 to separate the bit stream into information related to the decoding process and coded data related to coefficients, and decodes the coded data of the header part. More specifically, the separation decoding unit 202 supplies the quantization matrix coded data to the quantization matrix decoding unit 209, and supplies the coded data of the image to the decoding unit 203.

[0088] In S402, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix code data reproduced in S401. As a result, the quantization matrix decoding unit 209 reproduces one-dimensional difference matrices 1000 to 1002 shown in Figs. 10(a) to 10(c). The quantization matrix decoding unit 209 then performs scanning as shown in Fig. 9(a) on the one-dimensional difference matrix 1000, decodes according to the coding table in Fig. 11(a), reproduces the quantization matrices 800 to 802 in Fig. 8(a), and stores them. The quantization matrix decoding unit 209 also performs scanning as shown in Fig. 9(b) on the one-dimensional difference matrices 1001 and 1102, decodes according to the coding table in Fig. 11(a), and reproduces the quantization matrices 801 and 802 in Figs. 8(b) and 802 in Figs. 8(c), and stores them.

[0089] In S403, the control unit 250 controls the decoding unit 203 to decode the coded data separated in S401, reproduce the chrominance integration information and the prediction information, and reproduce the residual coefficient of the luminance component and the chrominance residual coefficient according to the chrominance integration information. As described above, it should be noted that the chrominance residual coefficient to be reproduced may be both Cb and Cr, or only Cb.

[0090] In S404, the control unit 250 controls the inverse quantization and inverse transform unit 204 to perform inverse quantization on the residual coefficients using the quantization matrix reproduced in S402 to generate orthogonal transform coefficients. The inverse quantization and inverse transform unit 204 further performs inverse orthogonal transform to reproduce the prediction error. In this embodiment, the quantization matrix to be used in the inverse quantization process is determined according to the color component of the subblock to be decoded. That is, the inverse quantization and inverse transform unit 204 performs inverse quantization using the quantization matrix of FIG. 8(a) for the residual coefficients of the Y component, the quantization matrix of FIG. 8(b) for the residual coefficients of the Cb component, and the quantization matrix of FIG. 8(c) for the residual coefficients of the Cr component. However, the quantization matrix to be used is not limited to this, and may be the same as the quantization matrix used on the encoding side.

[0091] However, when the chrominance integration information indicates that chrominance residual coefficient integration coding is used for the sub-block, the inverse quantization and inverse transform unit 204 inverse quantizes the residual coefficient of the Cb component using the quantization matrix of Fig. 8(b) to reproduce the orthogonal transform coefficient of the Cb component. Then, the inverse quantization and inverse transform unit 204 multiplies the reproduced orthogonal transform coefficient of the Cb component by (-1) and reproduces the result as the orthogonal transform coefficient of the Cr component.

[0092] In S405, the control unit 250 controls the image reproduction unit 205 to reproduce the image generated in S403. Specifically, the image reproduction unit 205 reproduces a predicted image based on the prediction information and with reference to the frame memory 206. At this time, the image reproduction unit 205 uses intra prediction such as horizontal prediction and vertical prediction, as in S305 on the encoding side. Then, the image reproduction unit 205 reproduces image data from the reproduced predicted image and the prediction error generated in S404, and stores the reproduced image data in the frame memory 206.

[0093] In S406, the control unit 250 determines whether the decoding of all basic blocks in the frame of interest has been completed. If the decoding has been completed, the process proceeds to S407. If there are any undecoded blocks, the process returns to S403 to decode the next basic block.

[0094] In S407, the control unit 250 controls the in-loop filter unit 207 to perform in-loop filter processing on the image data reproduced in S405, generate a filtered image, and then terminate the processing.

[0095] With the above configuration and operation, it is possible to decode the coded bit stream generated by the image coding device described above, i.e., for sub-blocks using chrominance residual coefficient integrated coding, into a bit stream in which quantization is controlled for each frequency component and subjective image quality is improved.

[0096] In this embodiment, when chrominance matrix information is included in the bitstream header, the decoding process will be as follows.

[0097] When the chrominance integration information of a certain sub-block (in the embodiment, the sub-block is defined as a basic block) of a certain picture indicates that chrominance residual coefficient integration coding is not used, the residual coefficients of the Cb component of the sub-block are inverse quantized using the quantization matrix 801 shown in FIG. 8(b), and the residual coefficients of the Cr component are inverse quantized using the quantization matrix 802 shown in FIG. 8(c).

[0098] On the other hand, when the chrominance integration information of a certain sub-block of a certain picture indicates that the chrominance residual coefficient integration coding is performed, which of the sub-blocks of the components Cb and Cr is to be coded depends on the chrominance matrix information stored in the header. The following is an explanation of this.

[0099] When the chrominance matrix information is 0, the residual coefficients of the sub-block of the component Cb out of the components Cb and Cr are inversely quantized using the quantization matrix 801 in Fig. 8(b) to obtain transform coefficients, which are then decoded. Since the component Cr does not exist, the inversely quantized transform coefficients of the component Cb are multiplied by "-1" to generate the transform coefficients of the component Cr, and then decoded.

[0100] Also, when the chrominance matrix information is 1, of the components Cb and Cr, the residual coefficients of the sub-block of the component Cr are inversely quantized using the quantization matrix 802 in Fig. 8(c) to obtain transform coefficients, which are then decoded. Since the component Cb does not exist, the value obtained by multiplying the inversely quantized transform coefficients of the component Cr by "-1" is generated as the transform coefficients of the component Cb, and is decoded.

[0101] Also, when the chrominance matrix information is 2, the residual coefficients of one of the sub-blocks of the components Cb and Cr (for example, component Cb) are inverse quantized using the quantization matrix 801 in Fig. 8(b) to obtain transform coefficients, which are then decoded. Since the other component Cr does not exist, the inverse quantized transform coefficients of one component are multiplied by "-1" to generate the transform coefficients of the other component, and decoded.

[0102] As a result, it is possible to decode a bitstream that realizes unique quantization control for sub-blocks using chrominance residual coefficient joint coding.

[0103] [Second embodiment] Although each processing unit included in the image encoding device and the image decoding device of the above embodiment has been described as being configured with hardware, the processing performed by each processing unit shown in these figures may be configured with a computer program.

[0104] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer applicable to the image encoding device and image decoding device according to the above embodiment.

[0105] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and ROM 503, and executes the processes described above as being performed by the image processing device according to the embodiment. That is, the CPU 501 functions as each of the processing units shown in FIGS.

[0106] The RAM 502 has an area for temporarily storing programs and data acquired from the outside via an external storage device 506 and an I / F (interface) 507. The RAM 502 is also used as a work area used when the CPU 501 executes various processes. The RAM 502 can be allocated as a frame memory, for example, or can provide various other areas as appropriate.

[0107] The ROM 503 stores setting data for the computer, a boot program, etc. The operation unit 504 is composed of a keyboard, a mouse, etc., and can be operated by a user of the computer to input various instructions to the CPU 501. The display unit 505 displays the results of processing by the CPU 501. The display unit 505 is composed of, for example, a liquid crystal display.

[0108] The external storage device 506 is a large-capacity information storage device, such as a hard disk drive. The external storage device 506 stores an OS (operating system) and computer programs (application programs) for causing the CPU 501 to realize the functions of each unit shown in Figures 1 and 2. Furthermore, the external storage device 506 may store each image data to be processed.

[0109] Computer programs and data stored in the external storage device 506 are loaded into the RAM 502 as appropriate under the control of the CPU 501, and become the subject of processing by the CPU 501. A network such as a LAN or the Internet, and other devices such as a projector or display device can be connected to the I / F 507, and the computer can obtain and transmit various information via this I / F 507. Reference numeral 508 denotes a bus that connects the above-mentioned components.

[0110] In the above configuration, when the device is powered on, the CPU 501 executes a boot program stored in the ROM 503, and loads the OS stored in the external storage device 506 into the RAM 502 and executes it. Then, under the control of the OS, the CPU 501 loads an application program related to encoding or decoding from the external storage device 506 into the RAM 502 and executes it. As a result, the CPU 501 functions as each processing unit in Fig. 1 or Fig. 2, and the device functions as an image encoding device or an image decoding device.

[0111] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions. [Industrial Applicability]

[0112] The present invention is applicable to an encoding device and a decoding device for encoding and decoding still and moving images, and is particularly applicable to an encoding method and a decoding method that use a quantization matrix. [Explanation of symbols]

[0113] 101...input terminal, 102...block division unit, 103...quantization matrix storage unit, 104...prediction unit, 105...transformation and quantization unit, 106...inverse quantization and inverse transformation unit, 107...image reproduction unit, 108...frame memory, 109...in-loop filter unit, 110...encoding unit, 111...integrated encoding unit, 112...output terminal, 113...quantization matrix encoding unit

Claims

1. An image decoding device that decodes encoded data of an image, comprising: A decoding means for decoding data corresponding to quantized transform coefficients, which are quantized transform coefficients, from the encoded data; a derivation means for deriving a prediction error from the decoded data by at least performing inverse quantization using a quantization matrix; having The prediction error of the chrominance component can be derived using the first mode or the second mode; the image comprises a plurality of color difference components; the first mode is a mode in which a sign of a first quantized transform coefficient associated with a first chrominance component among the plurality of chrominance components is omitted in the coded data, and the first chrominance component and a second chrominance component among the plurality of chrominance components are integrated; In the first mode, the deriving means derives a first prediction error for the first chrominance component using a quantized transform coefficient associated with a chrominance component different from the first chrominance component; In the first mode, the derivation means dequantizes the quantized transform coefficient used to derive the first prediction error of the first color difference component using a quantization matrix corresponding to the color difference component different from the first color difference component; the second mode is a mode in which signs of the first quantized transform coefficients are not omitted in the encoded data; the quantization matrix used for inverse quantization of the quantized transform coefficient used to derive the first prediction error when the first mode is used is the same as the quantization matrix used for inverse quantization of a quantized transform coefficient associated with a chrominance component other than the first chrominance component when the second mode is used; The prediction error is for a block partitioned by one or more partitioning methods, including ternary tree partitioning.

2. An image decoding device comprising:

2. In the second mode, data corresponding to the first quantized transform coefficient and data corresponding to a second quantized transform coefficient associated with the chrominance component different from the first chrominance component are decoded from the encoded data.

2. The image decoding device according to claim 1.

3. In the first mode, the deriving means derives the first prediction error by multiplying a value related to the color difference component different from the first color difference component by a predetermined value.

2. The image decoding device according to claim 1.

4. The decoding means further decodes the predetermined value.

4. The image decoding device according to claim 3.

5. 1. An image decoding method for decoding encoded data of an image, comprising the steps of: a decoding step of decoding data corresponding to quantized transform coefficients from the encoded data; a derivation step of deriving a prediction error from the decoded data by at least performing inverse quantization using a quantization matrix; having The prediction error of the chrominance component can be derived using the first mode or the second mode; the image comprises a plurality of color difference components; the first mode is a mode in which a sign of a first quantized transform coefficient associated with a first chrominance component among the plurality of chrominance components is omitted in the coded data, and the first chrominance component and a second chrominance component among the plurality of chrominance components are integrated; In the first mode, the deriving step derives a first prediction error for the first chrominance component using a quantized transform coefficient associated with a chrominance component other than the first chrominance component; In the derivation step in the first mode, the quantized transform coefficient used for deriving the first prediction error of the first color difference component is inversely quantized using a quantization matrix corresponding to the color difference component different from the first color difference component; the second mode is a mode in which signs of the first quantized transform coefficients are not omitted in the encoded data; the quantization matrix used for inverse quantization of the quantized transform coefficient used for deriving the first prediction error in the derivation step in the first mode is the same as the quantization matrix used for inverse quantization of the quantized transform coefficient associated with a chrominance component other than the first chrominance component in the derivation step in the second mode; The prediction error is for a block partitioned by one or more partitioning methods, including ternary tree partitioning.

2. An image decoding method comprising:

6. In the second mode, data corresponding to the first quantized transform coefficient and data corresponding to a second quantized transform coefficient associated with the chrominance component different from the first chrominance component are decoded from the encoded data.

6. The image decoding method according to claim 5.

7. In the derivation step in the first mode, the first prediction error is derived by multiplying a value related to the color difference component other than the first color difference component by a predetermined value.

6. The image decoding method according to claim 5.

8. 8. The image decoding method according to claim 7, further comprising an information decoding step of further decoding the predetermined value in the first mode.

9. A program causing a computer to function as each of the means of the image decoding device according to any one of claims 1 to 4.

Images