Image decoding device, decoding method, and storage medium

Abstract

The invention provides an image decoding apparatus, a decoding method, and a storage medium. The present invention suppresses a reduction in subjective image quality and improves the coding efficiency of residual coefficients of two chrominance components. To this end, the image encoding device of the present invention encodes image data comprising one luminance component and two chrominance components, and comprises: a quantization unit that orthogonally transforms each component of a block obtained by dividing an image to be encoded, quantizing the transformation coefficient of each component obtained according to the orthogonal transformation; and an encoding unit that encodes the residual coefficient obtained by the quantization unit. Wherein, in a case where the orthogonally transformed transform coefficients of the two chroma components are encoded in an integrated manner, the encoding unit quantizes the orthogonally transformed transform coefficients obtained by the integrated manner using a predetermined quantization matrix, and encodes the obtained residual coefficients.

Description

[0001] (This application is a divisional application of an application filed on April 2, 2020, with application number 202080041967.8 and invention name “Image encoding device, image decoding device, method and program”.) Technical Field

[0002] The present invention relates to an image encoding device, an image decoding device, a method and a program. Background Art

[0003] The High Efficiency Video Coding (HEVC) coding method (hereinafter referred to as "HEVC") is a coding scheme for compressing and recording moving images. In order to improve coding efficiency, HEVC adopts a basic block that is larger in size than a conventional macroblock (16×16 pixels). This large-sized basic block is called a coding tree unit (CTU) and has a maximum size of 64×64 pixels. The CTU is further divided into sub-blocks as units for prediction and conversion, etc.

[0004] In addition, HEVC uses a process called "quantization matrix" in which a set of orthogonally transformed coefficients (hereinafter referred to as "orthogonal transform coefficients") is weighted according to the frequency component. Reducing more high-frequency component data (where degradation is not easily perceived by the human eye) enables compression efficiency to be improved while maintaining image quality. Patent document 1 discloses a technology for encoding such a quantization matrix.

[0005] Recently, activities have been started to develop international standards for more efficient coding schemes as the successor to HEVC. ISO / IEC and ITU-T have established the Joint Video Experts Group (JVET) and are promoting standardization as the Versatile Video Coding (VVC) coding scheme (hereinafter referred to as "VVC"). In order to improve coding efficiency, a new coding method is being considered for VVC in which the quantized residual coefficients of the two chrominance components (Cb component and Cr component) are encoded together (hereinafter referred to as "comprehensive chrominance residual coefficient encoding").

[0006] List of existing technologies

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Laid-Open No. 2013-38758 Summary of the invention

[0009] Problem that the invention aims to solve

[0010] Like HEVC, VVC is also considering introducing a quantization matrix. However, the quantization matrix in HEVC takes quantization and inverse quantization processing of luminance (Y) and chrominance (Cb, Cr) components, and does not support the new coding method of integrated chrominance residual coefficient coding. Therefore, there is the following problem: when using integrated chrominance residual coefficient coding, quantization cannot be controlled according to frequency components, so the subjective image quality cannot be improved. Thus, the present invention is completed in order to solve the above-mentioned problem, and its purpose is to improve the subjective image quality by using a quantization matrix that is also compatible with integrated chrominance residual coefficient coding to achieve quantization processing.

[0011] Solutions for solving problems

[0012] In order to solve this problem, the image encoding device according to the present invention has the following configuration, for example.

[0013] That is, an image encoding device that encodes image data consisting of one luminance component and two chrominance components, the image encoding device comprising:

[0014] a quantization unit that performs an orthogonal transform on each component of a block obtained by dividing the image to be encoded, and quantizes a transform coefficient of each component obtained according to the orthogonal transform; and

[0015] an encoding unit, which encodes the residual coefficients obtained by the quantization unit,

[0016] In which, when encoding the transformation coefficients of the orthogonal transformation of the two chrominance components in an integrated manner, the encoding unit uses a predetermined quantization matrix to quantize the transformation coefficients of the orthogonal transformation obtained according to the integration, and encodes the residual coefficients obtained according to the quantization.

[0017] Effects of the Invention

[0018] According to the present invention, it is possible to suppress degradation of subjective image quality while improving the encoding efficiency of residual coefficients of two chrominance components.

[0019] Other features and advantages of the present invention will be apparent from the following description in conjunction with the accompanying drawings. Note that the same reference numerals in the accompanying drawings represent the same or similar components. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.

[0021] Figure 1 is a block diagram showing an image encoding device according to the first embodiment.

[0022] Figure 2 is a block diagram showing an image decoding apparatus according to the first embodiment.

[0023] Figure 3 is a flowchart showing an example of image encoding processing in the image encoding device according to the first embodiment.

[0024] Figure 4 is a flowchart showing an image decoding process in the image decoding device according to the first embodiment.

[0025] Figure 5 is a diagram showing a hardware configuration of a computer applied in the second embodiment.

[0026] Fig. 6A is a diagram showing an example of the data structure of a bit stream generated in the embodiment.

[0027] Figure 6B is a diagram showing an example of the data structure of a bit stream generated in the embodiment.

[0028] Fig. 7A is a diagram showing an example of sub-block partitioning according to an embodiment.

[0029] Figure 7B is a diagram showing an example of sub-block partitioning according to an embodiment.

[0030] Figure 7C is a diagram showing an example of sub-block partitioning according to an embodiment.

[0031] Fig.7D is a diagram showing an example of sub-block partitioning according to an embodiment.

[0032] Fig. 7E is a diagram showing an example of sub-block partitioning according to an embodiment.

[0033] Figure 7F is a diagram showing an example of sub-block partitioning according to an embodiment.

[0034] Fig. 8A is a diagram showing an example of a quantization matrix used in the embodiment.

[0035] Figure 8B is a diagram showing an example of a quantization matrix used in the embodiment.

[0036] Figure 8C is a diagram showing an example of a quantization matrix used in the embodiment.

[0037] Fig.8D is a diagram showing an example of a quantization matrix used in the embodiment.

[0038] Fig. 9Ais a diagram showing a method of scanning elements of a quantization matrix used in the embodiment.

[0039] Fig. 9B is a diagram showing a method of scanning elements of a quantization matrix used in the embodiment.

[0040] Fig. 10A : is a diagram showing a difference matrix of a quantization matrix generated in the embodiment.

[0041] Fig. 10B : is a diagram showing a difference matrix of a quantization matrix generated in the embodiment.

[0042] Fig. 10C : is a diagram showing a difference matrix of a quantization matrix generated in the embodiment.

[0043] Fig.11A is a diagram showing an example of a coding table for encoding a difference value of a quantization matrix.

[0044] Fig. 11B is a diagram showing an example of a coding table for encoding a difference value of a quantization matrix. DETAILED DESCRIPTION

[0045] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A plurality of features are described in the embodiments, but the present invention is not limited to requiring all such features, and a plurality of such features may be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and redundant descriptions thereof are omitted.

[0046] Figure 1 1 is a block diagram showing an image encoding device according to the present embodiment. The image encoding device includes a control unit 150 that controls the device as a whole. The control unit 150 includes a CPU, a ROM that stores programs executed by the CPU, and a RAM used as a work area of ​​the CPU. The image encoding device also includes an input terminal 101, a block division unit 102, a quantization matrix holding unit 103, a prediction unit 104, a transform / quantization unit 105, an inverse quantization / inverse transform unit 106, an image reconstruction unit 107, a frame memory 108, an in-loop filter unit 109, an encoding unit 110, an integrated encoding unit 111, an output terminal 112, and a quantization matrix encoding unit 113.

[0047] The input terminal 101 inputs the image data to be encoded, which has been generated by the image data source, in units of frames. The image data source may be any type of device, such as an image capturing device, and a file server or storage medium storing the image data to be encoded. The output terminal 112 outputs the encoded data to an output destination device, and the output destination device may also be a storage medium or a file server, etc.

[0048] The block division unit 102 divides the image of the input frame into a plurality of basic blocks, and sequentially outputs one of the basic blocks as a basic block to the prediction unit 104 in a subsequent stage.

[0049] The quantization matrix holding unit 103 generates a plurality of quantization matrices before encoding, and holds these quantization matrices in an internal memory (not shown). Although the method for generating the quantization matrix is ​​not particularly limited, the user may input the quantization matrix, the quantization matrix may be calculated based on the characteristics of the input image, or a quantization matrix specified in advance as an initial value may be used. The quantization matrix holding unit 103 according to the present embodiment generates and holds Fig. 8A The two-dimensional quantization matrix 800 corresponding to the orthogonal transformation of 8×8 pixels, and Figure 8B and Figure 8C Two-dimensional quantization matrices 801 and 802 corresponding to orthogonal transformation of 4×4 pixels are shown. Quantization matrix 800 is a quantization matrix for a luminance component, and quantization matrices 801 and 802 are quantization matrices for two chrominance components.

[0050] The prediction unit 104 determines sub-block division for the image data in the basic block unit, and generates predicted image data by performing intra-frame prediction (prediction within a frame in units of sub-blocks) and inter-frame prediction (i.e., prediction between frames), etc. The prediction unit 104 also calculates and outputs a prediction error in units of pixels based on the input image data and the predicted image data. Along with the prediction error, the prediction unit 104 also outputs information required for prediction, such as information related to sub-block division, prediction mode, and motion vector, etc. Hereinafter, the information required for prediction will be referred to as "prediction information".

[0051] The transform / quantization unit 105 obtains an orthogonal transform coefficient (orthogonally transformed residual coefficient) by performing an orthogonal transform on the prediction error input from the prediction unit 104 in units of subblocks. In addition, the transform / quantization unit 105 obtains a quantized residual coefficient (quantized orthogonal transform coefficient) by quantizing the orthogonal transform coefficient using the quantization matrix stored in the quantization matrix holding unit 103.

[0052] The inverse quantization / inverse transform unit 106 inputs the residual coefficient from the transform / quantization unit 105, inversely quantizes the residual coefficient using the quantization matrix stored in the quantization matrix holding unit 103, and reconstructs the orthogonal transform coefficient. The inverse quantization / inverse transform unit 106 also performs inverse orthogonal transform and reconstructs the prediction error.

[0053] The image reconstruction unit 107 generates predicted image data by appropriately referring to the frame memory 108 based on the prediction information output from the prediction unit 104. The image reconstruction unit 107 generates reconstructed image data by adding the prediction error from the inverse quantization / inverse transform unit 106 to the predicted image data, and stores the reconstructed image data in the frame memory 108.

[0054] The in-loop filter unit 109 performs in-loop filter processing, such as deblocking filtering and sample adaptive offset, on the reconstructed image stored in the frame memory 108 , and stores the filtered image data in the frame memory 108 again.

[0055] The encoding unit 110 generates encoded data by encoding the residual coefficient output from the transform / quantization unit 105 and the prediction information output from the prediction unit 104 , and outputs the encoded data to the integrated encoding unit 111 .

[0056] The quantization matrix encoding unit 113 generates quantization matrix encoded data by encoding the quantization matrix held in the quantization matrix holding unit 103 , and outputs the quantization matrix encoded data to the integrated encoding unit 111 .

[0057] The integrated coding unit 111 generates header code data including the quantization matrix code data from the quantization matrix coding unit 113. Then, the integrated coding unit 111 forms a bit stream by adding the code data output from the coding unit 110 after the header code data. Then, the integrated coding unit 111 outputs the formed bit stream via the output terminal 112.

[0058] A more detailed description of the operation of encoding an image in the image encoding device will be given below. The present embodiment adopts a configuration in which moving image data in a 4:2:0 color format is input from the input terminal 101 in units of frames, but the configuration may be a configuration in which still image data corresponding to one frame is input. In addition, in order to simplify the description, only the intra-frame prediction encoding process will be described in the present embodiment; however, the present embodiment is not limited to this and can also be applied to the inter-frame prediction encoding process. In addition, for the purpose of description, the present embodiment assumes that the block division unit 102 divides the image data input from the input terminal into basic blocks consisting of 8×8 pixels. In other words, the basic block of 8×8 pixels contains 8×8 pixels of the luminance (Y) component and 4×4 pixels of the chrominance (Cb and Cr) component. Note that this is only to facilitate understanding, and the configuration is not limited to the above values ​​(size).

[0059] A quantization matrix is ​​generated and encoded before encoding an image.

[0060] The quantization matrix holding unit 103 first generates and holds a quantization matrix. Specifically, the quantization matrix is ​​generated according to the size of the sub-block to be encoded and the type of prediction method. Fig. 7A As shown, a quantization matrix corresponding to a basic block of 8×8 pixels that is not divided into sub-blocks is generated. In other words, the quantization matrix holding unit 103 generates an 8×8 quantization matrix for the luminance (Y) component and a 4×4 quantization matrix for the chrominance (Cb and Cr) component. However, the generated quantization matrix is ​​not limited to this, and a quantization matrix corresponding to the shape of the sub-block such as 4×8, 8×4, etc. can be generated. The method for determining each element of the quantization matrix is ​​not particularly limited. For example, a predetermined initial value can be used, or the elements can be set separately. The matrix can also be generated according to the characteristics of the image.

[0061] The quantization matrix holding unit 103 holds the quantization matrix generated in this manner in an internal memory (not shown). Fig. 8A A quantization matrix 800 is shown for the Y component. Figure 8B Quantization matrix 801 for the Cb component is shown, Figure 8C 802 for the Cr component is shown. To simplify the description, it is assumed that the quantization matrix is ​​composed of 64 pixels (8×8) and 16 pixels (4×4), and each square in a bold frame represents an element of the quantization matrix. FIG. 8A to FIG. 8C The three quantization matrices shown are held in a two-dimensional shape, but the elements in the quantization matrix are of course not limited thereto. It is also possible to hold a plurality of quantization matrices for the same color component according to the size of the subblock or according to the type of prediction method described later (e.g., whether intra-frame prediction or inter-frame prediction is used). In general, the quantization matrix implements quantization processing according to human visual characteristics, so that, as FIG. 8A to FIG. 8C As shown, the elements of the low-frequency part corresponding to the upper left part of the quantization matrices 800 to 802 are small, while the elements of the high-frequency part corresponding to the lower right part are large.

[0062] The quantization matrix encoding unit 113 sequentially reads out the two-dimensional quantization matrix held in the quantization matrix holding unit 103, calculates the difference between the previous element and the next element obtained by scanning each element, and arranges the difference in a one-dimensional matrix. FIG. 8A to FIG. 8C The sizes of the quantization matrices 800 to 802 shown in FIG. 8 are shown in FIG. Fig. 9A or Fig. 9B to compute the difference between each element and the element immediately preceding it in the scan order. For example, using Fig. 9B Scanning method shown Figure 8C The quantization matrix 802 of 4×4 pixels shown in FIG. 1 is a quantization matrix of 4×4 pixels, and after the first element 8 located at the upper left, the element 14 immediately below it is scanned and the difference value +6 is calculated. In addition, when the first element (8 in the present embodiment) of the quantization matrix is ​​encoded, it is assumed that the difference with a predetermined initial value (e.g., 8) is calculated; however, the configuration is not limited thereto, and the difference with an arbitrary value or the value of the first element itself may be used. In short, the value may be an arbitrary value as long as the value is the same initial value as the initial value of the decoding device.

[0063] In this way, for Fig. 8A The quantization matrix 800 shown in FIG. 1 is used by the quantization matrix encoding unit 113. Fig. 9A The scanning method in generates the difference matrix. Similarly, for Figure 8B and Figure 8C The quantization matrices 801 and 802 shown in FIG. 1 are used by the quantization matrix encoding unit 113. Fig. 9B The scanning method in is used to generate the difference matrix. Fig. 10A Show according to Fig. 8A The difference matrix 1000 is obtained by quantizing the matrix 800. Fig. 10B Show according to Figure 8B The difference matrix 1002 obtained by quantizing the matrix 801 in FIG. Fig. 10C A difference matrix 1003 obtained from the quantization matrix 802 is shown.

[0064] The quantization matrix encoding unit 113 also generates quantization matrix encoding data by encoding each difference matrix generated as described above. Fig.11A The coding table shown in FIG. 1 is used to encode by adding codewords to each element of the difference matrix, and to generate quantization matrix encoded data. Note that the coding table is not limited to this, and for example, Fig. 11BIn this way, the quantization matrix encoding unit 113 outputs the generated quantization matrix encoded data to the integrated encoding unit 111 in the subsequent stage.

[0065] return Figure 1 , the integrated encoding unit 111 integrates the encoded data of the quantization matrix with the header information required to encode the image data.

[0066] Next, the image data is encoded. The image data corresponding to one frame input from the input terminal 101 is provided to the block division unit 102. The block division unit 102 divides the input image data of one frame into a plurality of basic blocks, and outputs the image data to the prediction unit 104 in units of basic blocks. In the present embodiment, the image data is provided to the prediction unit 104 in units of basic blocks of 8×8 pixels.

[0067] The prediction unit 104 performs prediction processing on the image data in units of basic blocks input from the block division unit 102. Specifically, subblock division for dividing the basic block into finer subblocks is determined, and an intra prediction mode such as horizontal prediction or vertical prediction is also determined in units of subblocks.

[0068] Will refer to 7A to 7F Describe the sub-block segmentation method. 7A to 7F The bold frames in blocks 700 to 705 of FIG. 7 represent the same size of 8×8 pixels as the size of the basic block. The quadrilaterals within the bold frames represent sub-blocks. Figure 7B An example of conventional square sub-block partitioning is shown, where an 8×8 pixel basic block 701 is partitioned into four 4×4 pixel sub-blocks. FIG. 7C to FIG. 7F An example of rectangular sub-block partitioning is shown. Figure 7C A basic block 702 is shown partitioned into two sub-blocks (longer in the vertical direction) of size 4x8 pixels. Fig.7D A basic block 703 is shown divided into two sub-blocks (longer in the horizontal direction) of size 8×4 pixels. Fig. 7E and Figure 7F In the basic blocks 704 and 705 of FIG. 1 , the block is divided into three rectangular sub-blocks at a ratio of 1:2:1, wherein the division directions are different. In this way, not only square sub-blocks but also rectangular sub-blocks are used for encoding processing.

[0069] Although this embodiment assumes that only Fig. 7A The block shown as not being divided into sub-blocks is used as a basic block having a size of 8×8 pixels, but the sub-block division method is not limited thereto. Figure 7B Quadtree partitioning such as the partitioning shown, such as Fig. 7E and Figure 7F The ternary tree partitioning such as the partitioning shown, or Figure 7Cand Fig.7D The binary tree partitioning shown is similar to the partitioning shown. When also using Fig. 7A In case of sub-block division other than the sub-block division shown in the figure, a quantization matrix corresponding to the sub-block to be used is generated in the quantization matrix holding unit 103. The generated quantization matrix is ​​also encoded by the quantization matrix encoding unit 113.

[0070] The prediction unit 104 generates predicted image data based on the determined prediction mode and the encoded area stored in the frame memory 108, and also calculates a prediction error in units of pixels from the input image data and the predicted image data, and outputs the prediction error to the transform / quantization unit 105. The prediction unit 104 also outputs information such as subblock division and prediction mode to the encoding unit 110 and the image reconstruction unit 107 as prediction information.

[0071] The transform / quantization unit 105 generates a residual coefficient by orthogonally transforming and quantizing the prediction error input from the prediction unit 104. In parallel with this, the transform / quantization unit 105 determines whether to use integrated chroma residual coefficient encoding that encodes the residual coefficients of the components Cb and Cr together.

[0072] The integrated chroma residual coefficient encoding used in the present embodiment will be described here. When there is a high correlation between the Cb residual coefficient and the Cr residual coefficient, the "integrated chroma residual coefficient encoding" reduces the amount of generated code and improves the encoding efficiency by encoding only one residual coefficient and omitting the encoding of the other residual coefficient. On the decoding side, only one of the encoded chroma residual coefficients is decoded, and the other chroma residual coefficient is reconstructed by calculation using the decoded chroma residual coefficient. In the present embodiment, for example, the integrated chroma residual coefficient encoding is performed only when the Cb residual coefficient and the Cr residual coefficient are in a negative correlation, and in the subsequent stage the encoding unit 110 only encodes the component Cb and omits the encoding of the Cr residual coefficient.

[0073] Return to Figure 1, the transform / quantization unit 105 first generates an orthogonal transform coefficient (orthogonally transformed residual coefficient) by performing an orthogonal transform process on the prediction error corresponding to the size of the sub-block. Next, the transform / quantization unit 105 determines whether to use integrated chroma residual coefficient encoding based on the correlation between the generated orthogonal transform coefficients of the components Cb and Cr. In other words, the transform / quantization unit 105 determines whether the orthogonal transform coefficient of the component Cb multiplied by (-1) is close to the orthogonal transform coefficient of the component Cr, and if it is determined that the coefficient is close, it is determined that the integrated chroma residual coefficient encoding will be used. The transform / quantization unit 105 also generates information indicating whether to use integrated chroma residual coefficient encoding (one bit is sufficient) as chroma integrated information. Then, the transform / quantization unit 105 quantizes the orthogonal transform coefficient according to the color component using the quantization matrix stored in the quantization matrix holding unit 103, and generates a residual coefficient (quantized orthogonal transform coefficient). This embodiment assumes Fig. 8A The quantization matrix is ​​used for the Y component orthogonal transform coefficients, Figure 8B The quantization matrix is ​​used for the orthogonal transform coefficients of the Cb component, Figure 8C The quantization matrix of is used for the Cr component orthogonal transform coefficient. However, the quantization matrix used is not limited to this. In addition, when it is determined that the integrated chrominance residual coefficient will be encoded, the quantization matrix of is used. Figure 8B The Cb component orthogonal transform coefficients are quantized by the quantization matrix of the quantization matrix, and the quantization of the Cr component orthogonal transform coefficients and the output to the subsequent stage are omitted. This makes it possible to reduce the amount of calculation by an amount corresponding to the quantization of the Cr component that is not used in the processing of the subsequent stage. The generated residual coefficients and chrominance integrated information are output to the encoding unit 110 and the inverse quantization / inverse transform unit 106.

[0074] The inverse quantization / inverse transform unit 106 reconstructs the orthogonal transform coefficients (more accurately, the residual coefficients that have not been quantized but have been orthogonally transformed) by using the quantization matrix stored in the quantization matrix holding unit 103 to inverse quantize the residual coefficients (quantized orthogonal transform coefficients) input from the transform / quantization unit 105. The inverse quantization / inverse transform unit 106 also performs an inverse orthogonal transform on the reconstructed orthogonal transform coefficients and reconstructs the prediction error. As with the transform / quantization unit 105, a quantization matrix corresponding to the color component of the sub-block to be encoded is used in the inverse quantization process. Specifically, the inverse quantization / inverse transform unit 106 uses the same quantization matrix as the quantization matrix used by the transform / quantization unit 105 to perform inverse quantization. In other words, Fig. 8A The quantization matrix is ​​used for the Y component residual coefficients, Figure 8B The quantization matrix is ​​used for the Cb component residual coefficients, Figure 8C The quantization matrix is ​​used for the Cr component residual coefficients.

[0075] In addition, when the chroma integrated information indicates that the sub-block is to be encoded using the integrated chroma residual coefficient, the inverse quantization / inverse transform unit 106 uses Figure 8B The quantization matrix 801 of the image processing unit 106 performs inverse quantization on the Cb component residual coefficient to reconstruct the Cb component orthogonal transformation coefficient. The inverse quantization / inverse transformation unit 106 multiplies the reconstructed Cb component orthogonal transformation coefficient by "-1" to reconstruct the Cr component orthogonal transformation coefficient. The prediction error reconstructed by inverse orthogonal transformation of the reconstructed orthogonal transformation coefficient is output to the image reconstruction unit 107.

[0076] The image reconstruction unit 107 reconstructs a predicted image by appropriately referring to the frame memory 108 based on the prediction information input from the prediction unit 104. Then, the image reconstruction unit 107 generates reconstructed image data based on the reconstructed predicted image and the prediction error reconstructed by the inverse quantization / inverse transform unit 106, and stores the reconstructed image data in the frame memory 108.

[0077] The in-loop filter unit 109 reads out the reconstructed image data from the frame memory 108 and performs in-loop filter processing such as deblocking filtering, etc. Then, the in-loop filter unit 109 stores the filtered image data in the frame memory 108 again.

[0078] The encoding unit 110 generates encoded data by entropy encoding the residual coefficients in units of sub-blocks generated by the transform / quantization unit 105, the chroma integrated information, and the prediction information input from the prediction unit 104. If the chroma integrated information indicates that the integrated chroma residual coefficients are not encoded for the corresponding sub-block, the encoding unit 110 encodes the residual coefficients of all color components of Y, Cb, and Cr. On the other hand, if the chroma integrated information indicates that the integrated chroma residual coefficients are encoded for the corresponding sub-block, the encoding unit 110 encodes the residual coefficients of the Y and Cb color components, but does not encode the residual coefficients of the Cr component. The entropy encoding method is not particularly specified, but Golomb encoding, arithmetic encoding, Huffman encoding, etc. can be used. The encoding unit 110 outputs the generated encoded data to the integrated encoding unit 111.

[0079] The integrated coding unit 111 forms a bit stream by multiplexing the coded data of the header and the coded data input from the coding unit 110. Then, the integrated coding unit 111 outputs the formed bit stream from the output terminal 112 to the outside (recording medium or network, etc.).

[0080] Fig. 6Ais an example of the data structure of the bit stream output in this embodiment. The sequence header contains the encoded data of the quantization matrix and is composed of the encoding results of each element. However, the encoded position is not limited to this, and of course a configuration for encoding the picture header part or other header part can be used. In addition, if the quantization matrix is ​​to be changed in the sequence, the matrix can be changed by newly encoding the quantization matrix. At this time, all quantization matrices can be rewritten, or part of the quantization matrix can be changed by specifying the quantization matrix color component corresponding to the quantization matrix to be rewritten.

[0081] Figure 3 1 is a flowchart showing encoding processing for one frame performed by the control unit 150 in the image encoding device of the embodiment.

[0082] First, before encoding an image, in S301, the control unit 150 controls the quantization matrix holding unit 103 to generate and hold a two-dimensional quantization matrix. Corresponding to an 8×8 pixel block, the quantization matrix holding unit 103 of this embodiment generates and holds FIG. 8A to FIG. 8C As shown, quantization matrices 800 to 802 correspond to the respective color components, namely, the Y component, the Cb component, and the Cr component.

[0083] In S302, the control unit 150 controls the quantization matrix encoding unit 113 to scan the quantization matrix generated and maintained in S301, calculate the difference between the elements, and generate a difference matrix. FIG. 8A to FIG. 8C The quantization matrices 800 to 802 shown use Fig. 9A or Fig. 9B The scanning method shown generates FIG. 10A to FIG. 10C The difference matrices 1001 to 1003 are shown. Then, the control unit 150 controls the quantization matrix encoding unit 113 to encode the generated difference matrix and generate quantization matrix encoded data.

[0084] In S303, the control unit 150 controls the integrated encoding unit 111 to encode and output the generated quantization matrix encoding data and the header information required for encoding the image data. In S304, the control unit 150 controls the block division unit 102 to divide the input image in units of frames into units of basic blocks. In S305, the control unit 150 controls the prediction unit 104 to perform prediction processing on the image data in units of the basic blocks generated in S304 to generate prediction information such as sub-block division information and prediction mode and predicted image data. In addition, the control unit 150 controls the prediction unit 104 to calculate the prediction error based on the input image data and the predicted image data.

[0085] In S306, the control unit 150 controls the transform / quantization unit 105 to perform an orthogonal transform on the prediction error calculated in S305 and generate a transform coefficient. In addition, the control unit 150 controls the transform / quantization unit 105 to quantize the quantization matrix generated and held in S301 and generate a residual coefficient. In parallel with this, the transform / quantization unit 105 is caused to determine whether to use integrated chrominance residual coefficient encoding, which encodes the Cb and Cr residual coefficients together. This embodiment assumes that Fig. 8A The quantization matrix is ​​used for the Y component orthogonal transform coefficients, Figure 8B The quantization matrix is ​​used for the orthogonal transform coefficients of the Cb component, Figure 8C The quantization matrix is ​​used for the Cr component orthogonal transform coefficients. In addition, when the integrated chrominance residual coefficients are to be encoded, the Figure 8B The quantization matrix is ​​used to quantize the Cb component orthogonal transform coefficients, and the quantization of the Cr component orthogonal transform coefficients is omitted.

[0086] Note that as a process for determining whether to use integrated chrominance residual coefficient encoding, the following specific process can be given as a process for determining whether the result of multiplying the orthogonal transformation coefficient of component Cb by (-1) is close to the orthogonal transformation coefficient of component Cr.

[0087] The orthogonal transformation coefficients of both the Cb component and the Cr component are 4×4. Here, Fig. 9B The transform coefficients of the scanning order are represented as Cb(i) and Cr(i) (where i=0, 1, ... 15). When the relationship between Cb(i) and Cr(i) can be considered as the following relationship for i=0, 1, 2, etc., Cr(i) is close to the value obtained by multiplying Cb(i) by "-1":

[0088] Cb(i)≈-Cr(i).

[0089] (Here, "≈" means that the two sides are approximately equal.)

[0090] In other words, as indicated by the following equation, this corresponds to the case where the sum of the chromaticities Cb and Cr is less than or equal to the threshold value:

[0091] |Cb(i)+Cr(i)|≤threshold.

[0092] Here, |x| represents the absolute value of the real number x.

[0093] In an embodiment, the number of chrominance component coefficients is 4×4, and therefore, if a predetermined threshold Th is used and the following equation is satisfied, it can be determined that the orthogonal transform coefficient of component Cb multiplied by (-1) is close to the orthogonal transform coefficient of component Cr:

[0094] Σ|Cb(i)+Cr(i)| <Th。

[0095] (Here, Σ represents the sum of i=0, 1, ...15)

[0096] Note that, as indicated by the following equation, instead of judging the absolute value, the judgment may be made based on the sum of the squares of the sums of the two chromaticities on the left side:

[0097] Σ{Cb(i)+Cr(i)} 2 <Th。

[0098] In the above embodiment, when a negative correlation is found between the residual coefficient of Cb and the residual coefficient of Cr, the integrated chrominance residual coefficient encoding is used, but it should be noted that the configuration is not limited to this. The correlation relationship can also be summarized as Cb(i)≈aCr(i) (where a is an integer). In this case, in the process for determining whether to use the integrated chrominance residual coefficient encoding, a condition such as the following is used:

[0099] Σ{|Cb(i)|-|Cr(i) / a|} <Th

[0100] or

[0101] Σ{|Cb(i)|-|Cr(i) / a|} 2 <Th。

[0102] When these conditions are met, integrated chroma residual coefficient coding is applied.

[0103] Note that information "a" indicating the correlation relationship may be encoded.

[0104] In S307, the control unit 150 controls the inverse quantization / inverse transform unit 106 to inverse quantize the residual coefficients generated in S306 using the quantization matrix generated and maintained in S301, and reconstruct the orthogonal transform coefficients. In this step, the same quantization matrix as that used in S306 is used for inverse quantization processing. In addition, when encoding the integrated chrominance residual coefficients, the inverse quantization matrix is ​​used. Figure 8B The Cb component residual coefficient is inversely quantized using the quantization matrix, and the Cb component orthogonal transformation coefficient is reconstructed. In addition, the reconstructed Cb component orthogonal transformation coefficient is multiplied by "-1" to reconstruct the Cr component orthogonal transformation coefficient. Then, the reconstructed orthogonal transformation coefficient is inversely orthogonally transformed, and the prediction error is reconstructed.

[0105] In S308 , the control unit 150 controls the image reconstruction unit 107 to reconstruct a predicted image based on the prediction information generated in S305 , reconstruct image data from the reconstructed predicted image and the prediction error generated in S307 , and store the image data in the frame memory 108 .

[0106] In S309, the control unit 150 controls the encoding unit 110 to encode the prediction information generated in S305 and the residual coefficients and chroma integrated information generated in S306, and generate encoded data. Here, if the chroma integrated information indicates that the integrated chroma residual coefficient encoding is not used for the corresponding sub-block, the encoding unit 110 encodes the residual coefficients of all color components of Y, Cb and Cr. On the other hand, if the chroma integrated information indicates that the integrated chroma residual coefficient encoding will be used for the corresponding sub-block, the encoding unit 110 encodes the residual coefficients of the Y and Cb color components, but omits the encoding of the residual coefficient of the Cr component. When the integrated chroma residual coefficient encoding is used, the average value of the sign-reversed value of the residual coefficient of the Cb component and the residual coefficient of the Cr component can be encoded as the encoded data of the color component instead of encoding the residual coefficient of the Cb component. In this case, the degradation of the color reproduction of the decoded image can be suppressed. In addition, the encoding unit 110 outputs the generated encoded data to the integrated encoding unit 111. The integrated encoding unit 111 positions the encoded data from the encoding unit 110 to follow the previously generated header, and outputs the data.

[0107] In S310, the control unit 150 determines whether encoding has been completed for all basic blocks in the frame of interest. If the control unit 150 determines that encoding has been completed, the sequence moves to step S311, and if the control unit 150 determines that there are unencoded basic blocks left, the sequence returns to S304, and encoding continues from the next basic block.

[0108] In S311 , the control unit 150 controls the in-loop filter unit 109 to perform in-loop filter processing on the image data reconstructed in S308 and generate a filtered image, and ends the processing.

[0109] The above configuration and operation, particularly S306, enable controlling the quantization of each frequency component and improving subjective image quality by quantizing using a quantization matrix for a subblock encoded using an integrated chrominance residual coefficient.

[0110] Although this embodiment describes the use of integrated chroma residual coefficient encoding Figure 8B The configuration of the quantization matrix of , but the quantization matrix used is not limited to this. For example, it can be Figure 8C The same effect can be achieved by configuring the quantization matrix for Cr component quantization and inverse quantization. Figure 8B or Figure 8C The existing quantization matrix is ​​used to generate a new quantization matrix to configure quantization and inverse quantization.

[0111] In addition, a configuration in which different quantization matrices are encoded for the integrated chrominance residual coefficient encoding can also be used. FIG. 8A to FIG. 8C In addition to the existing quantization matrices 800 to 802 in Fig.8D The quantization matrix 803 for integrated chroma residual coefficient coding is used for coding, and the quantization matrix 803 is used in the quantization of the sub-blocks coded using the integrated chroma residual coefficients. This enables independent quantization control of the sub-blocks coded using the integrated chroma residual coefficients.

[0112] Furthermore, although the present embodiment describes a configuration in which a quantization matrix for a subblock encoded using an integrated chrominance residual coefficient is uniquely determined, a configuration in which a quantization matrix can be selected by introducing an identifier may also be used. For example, Figure 6B It is shown that by newly introducing chrominance matrix information (assumed to be two bits) for specifying the quantization matrix, the quantization matrix is ​​selective for the sub-block encoded using the integrated chrominance residual coefficients.

[0113] For example, when the chroma matrix information is 0, the sub-block encoded using the integrated chroma residual coefficient is used Figure 8B The quantization matrix of the Cb component is shown in FIG. When the chrominance matrix information is 1, the sub-block encoded using the integrated chrominance residual coefficient is used. Figure 8C On the other hand, when the chrominance matrix information is 2, the sub-block encoded using the integrated chrominance residual coefficient is encoded using a separate encoding. Fig.8D This enables selective reduction of the quantization matrix code size and unique quantization control of sub-blocks encoded using integrated chrominance residual coefficients.

[0114] To describe the above in more detail, when the chroma synthesis information of a given sub-block (in this embodiment, sub-block = basic block) in a given picture indicates that the integrated chroma residual coefficient encoding is not used, the Figure 8B The quantization matrix 801 shown in FIG. 8 quantizes the transform coefficients of the component Cb in the sub-block, and uses Figure 8C The quantization matrix 802 in quantizes the transform coefficients of the Cr component.

[0115] On the other hand, when the chroma integration information of a given subblock of a given picture indicates that the integrated chroma residual coefficient will be used for encoding, which subblock of the components Cb and Cr will be encoded depends on the chroma matrix information stored in the header. For example, when the chroma matrix information of the components Cb and Cr is 0, the integrated chroma residual coefficient will be used. Figure 8B The quantization matrix 801 of the component Cb is used to quantize and encode the transform coefficients of the sub-block, while omitting the quantization and encoding of the component Cr. In addition, when the chrominance matrix information of the components Cb and Cr is 1, the quantization matrix 801 of the component Cb is used to quantize and encode the transform coefficients of the sub-block, while omitting the quantization and encoding of the component Cr. Figure 8CThe quantization matrix 802 of the component Cr is used to quantize and encode the transform coefficients of the sub-block of the component Cr, while omitting the quantization and encoding of the component Cb (the component Cb can be encoded, and the component Cr can be omitted). Then, when the chrominance matrix information of the components Cb and Cr is 2, the quantization matrix 802 of the component Cr is used to quantize and encode the transform coefficients of the sub-block of the Fig.8D The quantization matrix 803 quantizes and encodes the transform coefficients of the sub-block of component Cb (or component Cr), while omitting quantization and encoding of component Cr (or component Cb).

[0116] Note that when using Fig.8D When the quantization matrix 803 of the quantization matrix is ​​generated, the encoded data of the quantization matrix 803 is stored in the header. In other words, the quantization matrix holding unit 103 generates and holds the quantization matrix 803, and the quantization matrix encoding unit 113 also encodes the quantization matrix 803.

[0117] Figure 2 1 is a block diagram showing an image decoding device according to the present embodiment that decodes the encoded image data generated by the above-mentioned image encoding device. The configuration and operation related to the decoding process will be described with reference to the drawings.

[0118] The image decoding device includes a control unit 250 that controls the entire device. The control unit 250 includes a CPU, a ROM that stores programs executed by the CPU, and a RAM used as a work area for the CPU. The image decoding device also includes an input terminal 201, a separation / decoding unit 202, a decoding unit 203, an inverse quantization / inverse transformation unit 204, an image reconstruction unit 205, a frame memory 206, an in-loop filter unit 207, an output terminal 208, and a quantization matrix decoding unit 209.

[0119] The input terminal 201 inputs a coded bit stream, and although the input source is, for example, a storage medium storing the coded stream, the type of the input source is not limited, and the stream may be input from a network.

[0120] The separation / decoding unit 202 separates information related to the decoding process and coded data related to the coefficients from the bit stream, and also decodes the coded data present in the header portion of the bit stream. The separation / decoding unit 202 of this embodiment separates the quantization matrix coded data and outputs the data to the quantization matrix decoding unit 209. The separation / decoding unit 202 also outputs the coded data of the image to the decoding unit 203. In other words, the separation / decoding unit 202 performs the decoding process with the quantization matrix coded data. Figure 1 The operation of the integrated encoding unit 111 is the opposite operation.

[0121] The quantization matrix decoding unit 209 reconstructs and holds the quantization matrix by decoding the quantization matrix encoded data supplied from the separation / decoding unit 202 .

[0122] The decoding unit 203 decodes the encoded data of the image input from the separation / decoding unit 202, and reconstructs the residual coefficient, chrominance integrated information, and prediction information.

[0123] and Figure 1 Likewise, the inverse quantization / inverse transform unit 106 of the inverse quantization / inverse transform unit 204 performs inverse quantization on the residual coefficient by using the reconstructed quantization matrix, obtains an inverse quantization coefficient, and also performs inverse orthogonal transform to reconstruct the prediction error.

[0124] The image reconstruction unit 205 generates predicted image data based on the input prediction information by appropriately referring to the frame memory 206. Then, the image reconstruction unit 205 generates reconstructed image data from the predicted image data and the prediction error reconstructed by the inverse quantization / inverse transform unit 204, and stores the reconstructed image data in the frame memory 206.

[0125] and Figure 1 Similarly, the in-loop filter unit 109 of the in-loop filter unit 207 performs in-loop filter processing such as deblocking filtering on the reconstructed image data stored in the frame memory 206, and stores the filtered image data in the frame memory again.

[0126] The output terminal 208 sequentially outputs the frame images stored in the frame memory 206 to the outside. Although the output destination is generally a display device, other devices may be used instead.

[0127] The operation related to decoding an image performed by the image decoding apparatus of the embodiment described above will be described in further detail.The present embodiment employs a configuration in which a coded bit stream is input in units of frames.

[0128] exist Figure 2 In the embodiment, a bit stream corresponding to one frame input from the input terminal 201 is supplied to the separation / decoding unit 202. The separation / decoding unit 202 separates information related to the decoding process and coded data related to coefficients from the bit stream, and decodes the coded data present in the header portion of the bit stream. Then, the separation / decoding unit 202 supplies the quantization matrix coded data included in the header portion to the quantization matrix decoding unit 209, and supplies the coded data of the image data to the decoding unit 203. Specifically, the separation / decoding unit 202 first separates the quantization matrix coded data from the header portion, and supplies the quantization matrix decoding data to the decoding unit 203. Fig. 6A The quantization matrix encoding data is extracted from the sequence header of the bit stream shown in FIG. 1 and the quantization matrix encoding data is output to the quantization matrix decoding unit 209. In this embodiment, the quantization matrix encoding data is extracted and outputted. FIG. 8A to FIG. 8C Then, the quantization matrix coded data corresponding to the quantization matrix shown in FIG.

[0129] The quantization matrix decoding unit 209 first decodes the input quantization matrix encoded data and then reconstructs FIG. 10A to FIG. 10C The one-dimensional difference matrices 1000 to 1002 are shown. Then, as on the encoding side, the quantization matrix decoding unit 209 uses Fig.11A The coding table shown decodes each quantization matrix. Although the embodiment assumes the use of Fig.11A The code table shown, but you can use it instead Fig. 11B In short, it is sufficient to use the same coding table as that used on the encoding side. The quantization matrix decoding unit 209 reversely scans the reconstructed one-dimensional difference matrix 1000 to 1002, and reconstructs and holds the two-dimensional quantization matrix. The inverse operation of the operation performed by the quantization matrix encoding unit 113 on the encoding side is performed here. In other words, the quantization matrix encoding unit 113 of this embodiment uses Fig. 9A or Fig. 9B The scanning methods shown are based on their respective sizes. FIG. 10A to FIG. 10C The difference matrix 1000 to 1002 shown is decoded. As a result, FIG. 8A to FIG. 8C As shown, the quantization matrix encoding unit 113 reconstructs and holds three types of quantization matrices 800 to 802.

[0130] The decoding unit 203 decodes the coded data provided from the separation / decoding unit 202, reconstructs the chroma integrated information and the prediction information, and also reconstructs the residual coefficient based on the reconstructed chroma integrated information. The decoding unit 203 first reconstructs the prediction information and obtains the prediction mode for the sub-block. Next, the decoding unit 203 reconstructs the chroma integrated information and determines whether the integrated chroma residual coefficient encoding is used for the sub-block. If the chroma integrated information indicates that the integrated chroma residual coefficient encoding is not used for the corresponding sub-block, the decoding unit 203 decodes and reconstructs the residual coefficients of all color components of Y, Cb and Cr. On the other hand, if the chroma integrated information indicates that the integrated chroma residual coefficient encoding is used for the corresponding sub-block, the decoding unit 203 decodes the residual coefficients of the Y and Cb color components (there is no coded data for the Cr color component, so the corresponding decoding process is not performed). The decoding unit 203 outputs the reconstructed chroma integrated information and the residual coefficient to the inverse quantization / inverse transformation unit 204, and outputs the reconstructed prediction information to the image reconstruction unit 205.

[0131] The inverse quantization / inverse transform unit 204 generates an orthogonal transform coefficient by inverse quantizing the input residual coefficient using the quantization matrix reconstructed by the quantization matrix decoding unit 209, and also reconstructs the prediction error by performing inverse orthogonal transform. Like the inverse quantization / inverse transform unit 106 on the encoding side, the inverse quantization / inverse transform unit 204 performs inverse quantization using a quantization matrix corresponding to the color component of the subblock to be decoded. In other words, Fig. 8AThe quantization matrix is ​​used for the Y component residual coefficients, Figure 8B The quantization matrix is ​​used for the Cb component residual coefficients, Figure 8C The quantization matrix of is used for the Cr component residual coefficient. However, if the chroma integrated information indicates that integrated chroma residual coefficient encoding is used in the corresponding sub-block, the residual coefficient of the orthogonal transform coefficient of the Cr component is not provided from the decoding unit 203. Therefore, the inverse quantization / inverse transform unit 204 uses Figure 8B The residual coefficients of the Cb component are inversely quantized using the quantization matrix, and then the orthogonal transform coefficients of the Cr component are reconstructed. In other words, the processing efficiency is improved by using the quantization matrix for inverse quantization before reconstructing the Cr component. Specifically, the inverse quantization / inverse transform unit 204 multiplies the reconstructed Cb component orthogonal transform coefficients by (-1) to reconstruct the Cr component orthogonal transform coefficients, and provides these coefficients to the image reconstruction unit 205. The prediction error reconstructed by inverse orthogonal transforming the reconstructed orthogonal transform coefficients is output to the image reconstruction unit 205. However, the quantization matrix used is not limited to this, and any quantization matrix can be used as long as the quantization matrix is ​​the same as the quantization matrix used by the transform / quantization unit 105 and the inverse quantization / inverse transform unit 106 on the encoding side.

[0132] Although the case of a negative correlation relationship is described above as an example, the configuration is not limited thereto.

[0133] If the correlation relationship of Cb(i)≈aCr(i) (where a is an integer) is observed and the integrated chrominance residual coefficient encoding is performed, the orthogonal transform coefficient of the Cb component multiplied by "a" will be reconstructed as the orthogonal transform coefficient of the Cr component. The information "a" indicating the correlation relationship may be decoded from the bit stream, or the information "a" indicating the correlation relationship may be held in advance as a rule between the encoding side and the decoding side.

[0134] The image reconstruction unit 205 reconstructs a predicted image by appropriately referring to the frame memory 206 based on the prediction information input from the decoding unit 203. The image reconstruction unit 205 of the present embodiment uses intra-frame prediction such as horizontal prediction or vertical prediction, similarly to the prediction unit 104 on the encoding side. The specific prediction process is the same as the prediction process of the prediction unit 104 on the encoding side, and therefore will not be described. The image reconstruction unit 205 generates image data based on the predicted image and the prediction error input from the inverse quantization / inverse transform unit 204, and stores the image data in the frame memory 206. The stored image data is used for reference during prediction.

[0135] In the same manner as the in-loop filter unit 109 on the encoding side, the in-loop filter unit 207 reads the reconstructed image from the frame memory 206 and performs in-loop filter processing such as deblocking filtering, etc. The in-loop filter unit 207 then stores the filtered image in the frame memory 206 again.

[0136] The reconstructed image stored in the frame memory 206 is finally output from the output terminal 208 to the outside (a typical example of the outside is a display device).

[0137] Figure 4 : is a flowchart showing the decoding process performed by the control unit 250 in the image decoding device according to the present embodiment.

[0138] First, in S401, the control unit 250 controls the separation / decoding unit 202 to separate information related to the decoding process and the coded data related to the coefficients from the bit stream, and decodes the coded data of the header part. More specifically, the separation / decoding unit 202 provides the quantization matrix coded data to the quantization matrix decoding unit 209, and provides the coded data of the image to the decoding unit 203.

[0139] In S402, the control unit 250 controls the quantization matrix decoding unit 209 to decode the quantization matrix encoded data reconstructed in S401. As a result, the quantization matrix decoding unit 209 reconstructs FIG. 10A to FIG. 10C The one-dimensional difference matrices 1000 to 1002 are shown in FIG. Then, the quantization matrix decoding unit 209 is as follows Fig. 9A Scanning the one-dimensional difference matrix 1000 as shown, according to Fig.11A The encoding table is decoded, reconstructed and maintained Fig. 8A Quantization matrix 800. In addition, the quantization matrix decoding unit 209 is as follows Fig. 9B Scan the one-dimensional difference matrices 1001 and 1002 as shown, according to Fig.11A The encoding table is decoded, reconstructed and maintained Figure 8B and Figure 8C Quantization matrices 801 and 802 in .

[0140] In S403, the control unit 250 controls the decoding unit 203 to decode the coded data separated in S401, reconstruct the chrominance synthesis information and the prediction information, and reconstruct the residual coefficient of the luminance component and the chrominance residual coefficient according to the chrominance synthesis information. Note that, as described earlier, the chrominance residual coefficient to be reconstructed may be both Cb and Cr, or only Cb.

[0141] In S404, the control unit 250 controls the inverse quantization / inverse transform unit 204 to inverse quantize the residual coefficients using the quantization matrix reconstructed in S402 and generate orthogonal transform coefficients. The inverse quantization / inverse transform unit 204 also performs inverse orthogonal transform and reconstructs the prediction error. In this embodiment, the quantization matrix to be used in the inverse quantization is determined according to the color component of the sub-block to be decoded. In other words, the inverse quantization / inverse transform unit 204 uses Fig. 8A The quantization matrix is ​​used to inversely quantize the Y component residual coefficients, using Figure 8B The quantization matrix is ​​used to inversely quantize the residual coefficients of the Cb component, using Figure 8C The quantization matrix of is used to inversely quantize the Cr component residual coefficient. However, the quantization matrix used is not limited to this, and any quantization matrix can be used as long as the quantization matrix is ​​the same as the quantization matrix used on the encoding side.

[0142] However, when the chroma integration information indicates that the integrated chroma residual coefficients are used for encoding in the sub-block, the inverse quantization / inverse transform unit 204 uses Figure 8B The Cb component residual coefficient is inversely quantized by the quantization matrix of to reconstruct the Cb component orthogonal transformation coefficient. Then, the inverse quantization / inverse transformation unit 204 multiplies the reconstructed Cb component orthogonal transformation coefficient by (-1) to reconstruct the Cr component orthogonal transformation coefficient.

[0143] In S405, the control unit 250 controls the image reconstruction unit 205 to reconstruct the image generated in S403. Specifically, the image reconstruction unit 205 reconstructs the predicted image by referring to the frame memory 206 based on the prediction information. At this time, the image reconstruction unit 205 uses intra-frame prediction such as horizontal prediction or vertical prediction, as in S305 on the encoding side. Then, the image reconstruction unit 205 reconstructs the image data based on the reconstructed predicted image and the prediction error generated in S404, and stores the reconstructed image data in the frame memory 206.

[0144] In S406, the control unit 250 determines whether decoding has been completed for all basic blocks in the frame of interest. If not, the sequence moves to S407, and if there are undecoded blocks left, the sequence returns to S403 to decode the next basic block.

[0145] In S407 , the control unit 250 controls the in-loop filter unit 207 to perform in-loop filter processing on the image data reconstructed in S405 and generate a filtered image, and then ends the processing.

[0146] Even for an encoded bitstream generated using the above-described image encoding apparatus (ie, a subblock encoded using an integrated chroma residual coefficient), the above-described configuration and operation enable decoding of a bitstream with improved subjective image quality by controlling quantization for each frequency component.

[0147] Note that in this embodiment, the following decoding process is performed for the case where the header of the bit stream contains chroma matrix information.

[0148] When the chroma integration information of a given sub-block (in this embodiment, sub-block = basic block) in a given picture indicates that the integrated chroma residual coefficient encoding is not used, the Figure 8B The quantization matrix 801 shown in FIG. 8 performs inverse quantization on the residual coefficients of the component Cb in the sub-block, and uses Figure 8C The quantization matrix 802 inversely quantizes the residual coefficients of the Cr component.

[0149] On the other hand, when the chroma integrated information of a given subblock of a given picture indicates encoding using integrated chroma residual coefficients, which subblock of components Cb and Cr will be encoded depends on the chroma matrix information stored in the header. This case will be described below.

[0150] When the chrominance matrix information is 0, the residual coefficients of the sub-blocks of component Cb in component Cb and Cr are calculated by using Figure 8B The quantization matrix 801 of Cr is inversely quantized to obtain a transform coefficient and is decoded. Since there is no component Cr, a value obtained by multiplying the inversely quantized transform coefficient of the component Cb by "-1" is generated and decoded as a transform coefficient of the component Cr.

[0151] In addition, when the chrominance matrix information is 1, the residual coefficients of the sub-blocks of component Cr in components Cb and Cr are calculated by using Figure 8C The quantization matrix 802 of Cr is inversely quantized to obtain a transform coefficient and is decoded. Since the component Cb does not exist, a value obtained by multiplying the inversely quantized transform coefficient of the component Cr by "-1" is generated and decoded as a transform coefficient of the component Cb.

[0152] In addition, when the chrominance matrix information is 2, the residual coefficient of the sub-block of one component (for example, component Cb) of components Cb and Cr is calculated by using Figure 8B The quantization matrix 801 of 804 is inversely quantized to obtain a transform coefficient and is decoded. Since the other component Cr does not exist, a value obtained by multiplying the inversely quantized transform coefficient of one component by "-1" is generated and decoded as a transform coefficient of the other component.

[0153] As a result, it is possible to decode a bit stream in which independent quantization control is achieved for sub-blocks encoded using integrated chrominance residual coefficients.

[0154] Second embodiment

[0155] The processing units of the image encoding device and the image decoding device in the above-mentioned embodiments are described as being constituted by hardware. However, the processing performed by the processing units shown in the drawings may be constituted by a computer program.

[0156] Figure 5 : is a block diagram showing an example of the hardware configuration of a computer to which the image encoding device and the image decoding device according to the above-described embodiments can be applied.

[0157] The CPU 501 controls the entire computer using computer programs and data stored in the RAM 502 and the RAM 503, and executes the above-described processing as the processing performed by the image processing apparatus according to the above-described embodiment. In other words, the CPU 501 functions as Figure 1 and Figure 2 The processing unit shown.

[0158] The RAM 502 has an area for temporarily storing programs and data obtained from the outside via an external storage device 506 and an I / F (interface) 507. In addition, the RAM 502 is also used as a work area when the CPU 501 performs various types of processing. For example, the RAM 502 can be allocated as a frame memory, or various other areas can be appropriately provided.

[0159] Configuration data and a boot program of the computer and the like are stored in the ROM 503. The operation unit 504 is constituted by a keyboard and a mouse and the like, and by manipulating the operation unit 504, the user of the computer can input various types of instructions to the CPU 501. The display unit 505 displays the processing result of the central processor 501. The display unit 505 is constituted by, for example, a liquid crystal display.

[0160] The external storage device 506 is a high-capacity information storage device represented by a hard disk drive device. The external storage device 506 stores an OS (operating system) and a program for the CPU 501 to implement the Figure 1 and Figure 2 The computer program (application program) etc. which implements the functions of each unit shown in the figure. The image data to be processed may also be stored in the external storage device 506.

[0161] Computer programs and data and the like stored in the external storage device 506 are appropriately loaded into the RAM 502 under the control of the CPU 501 and then processed by the CPU 501. A network such as a LAN and the Internet and other devices such as a projection device and a display device can be connected to the I / F 507, and the computer can obtain and send various information via the I / F 507. 508 denotes a bus that connects the above-mentioned units to each other.

[0162] In the above configuration, when the power of the device is turned on, the CPU 501 executes the boot program stored in the ROM 503, loads the OS stored in the external storage device 506 into the RAM 502, and executes the OS. Then, under the control of the OS, the CPU 501 loads the application program for encoding or decoding from the external storage device 506 into the RAM 502 and executes the program. As a result, the CPU 501 functions as Figure 1 or Figure 2 The processing units shown, therefore, the device is used as an image encoding device or an image decoding device.

[0163] Other embodiments

[0164] The present invention can be implemented by providing a program for implementing one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and causing one or more processors in a computer of the system or device to read out and execute the program. The present invention can also be implemented by a circuit (e.g., ASIC) for implementing one or more functions.

[0165] Industrial Applicability

[0166] The present invention can be used for an encoding device and a decoding device for encoding and decoding still images and moving images. The present invention is particularly suitable for an encoding scheme and a decoding scheme using a quantization matrix.

[0167] The present invention is not limited to the above-described embodiments, and various changes and modifications may be made within the spirit and scope of the present invention. Therefore, the following claims are attached to inform the public of the scope of the present invention.

[0168] This application claims the benefit of Japanese Patent Application No. 2019-107454, filed on Jun. 7, 2019, which is hereby incorporated by reference herein in its entirety.

Claims

1. An image decoding device for decoding coded data of an image, comprising: a decoding unit configured to decode data corresponding to the quantized transform coefficients from the encoded data; a deriving unit configured to derive a prediction error from the decoded data by performing at least inverse quantization using a quantization matrix; as well as an execution unit configured to perform a sample adaptive offset process on the decoded image, wherein the prediction error corresponding to the chrominance component can be derived using the first mode or the second mode, Wherein, the image includes multiple chrominance components, The first mode is a mode in which the encoded data of the first quantized transform coefficient associated with the first chroma component among the plurality of chroma components is omitted in the encoded data, and the first chroma component and the second chroma component among the plurality of chroma components are integrated, wherein, in the first mode, the deriving unit is configured to derive a first prediction error of the first chrominance component using a quantized transform coefficient associated with a chrominance component different from the first chrominance component, wherein, in the first mode, the deriving unit is configured to inverse quantize the quantized transform coefficients used to derive the first prediction error of the first chrominance component using a quantization matrix corresponding to a chrominance component different from the first chrominance component, The second mode is a mode of the encoded data in which the first quantized transform coefficient is not omitted in the encoded data, and In which, the quantization matrix used to inverse quantize the quantized transform coefficients used to derive the first prediction error when using the first mode is the same as the quantization matrix used to inverse quantize the quantized transform coefficients associated with chrominance components different from the first chrominance components when using the second mode.

2. The image decoding device according to claim 1, wherein: In the second mode, data corresponding to the first quantized transform coefficient and data corresponding to a second quantized transform coefficient associated with a chroma component different from the first chroma component are decoded from the encoded data.

3. The image decoding device according to claim 1, wherein: In the first mode, the deriving unit is configured to derive the first prediction error by multiplying a value related to a chroma component other than the first chroma component by a predetermined value.

4. A method for decoding coded data of an image, comprising: decoding data corresponding to quantized transform coefficients from the encoded data; deriving a prediction error from the decoded data by at least inverse quantization using a quantization matrix; as well as performing a sample adaptive offset process on the decoded image, wherein the prediction error corresponding to the chrominance component can be derived using the first mode or the second mode, Wherein, the image includes multiple chrominance components, The first mode is a mode in which the encoded data of the first quantized transform coefficient associated with the first chroma component among the plurality of chroma components is omitted in the encoded data, and the first chroma component and the second chroma component among the plurality of chroma components are integrated, wherein, in the derivation in the first mode, a first prediction error of the first chrominance component is derived using quantized transform coefficients associated with a chrominance component different from the first chrominance component, wherein, in the derivation in the first mode, the quantized transform coefficients used to derive the first prediction error of the first chrominance component are inversely quantized using a quantization matrix corresponding to a chrominance component different from the first chrominance component, The second mode is a mode of the encoded data in which the first quantized transform coefficient is not omitted in the encoded data, and In which, the quantization matrix used to inverse quantize the quantized transform coefficients used to derive the first prediction error when using the first mode is the same as the quantization matrix used to inverse quantize the quantized transform coefficients associated with chrominance components different from the first chrominance components when using the second mode.

5. The method according to claim 4, wherein: In the second mode, data corresponding to the first quantized transform coefficient and data corresponding to a second quantized transform coefficient associated with a chroma component different from the first chroma component are decoded from the encoded data.

6. The method according to claim 4, wherein: In the first mode, the first prediction error is derived by multiplying a value related to a chroma component other than the first chroma component by a predetermined value.

7. A non-transitory computer-readable storage medium storing a computer program, which, when read and executed by a computer, causes the computer to perform the steps of a method for decoding coded data of an image, wherein: The method comprises: decoding data corresponding to quantized transform coefficients from the encoded data; deriving a prediction error from the decoded data by at least inverse quantization using a quantization matrix; and performing a sample adaptive offset process on the decoded image, wherein the prediction error corresponding to the chrominance component can be derived using the first mode or the second mode, Wherein, the image includes multiple chrominance components, The first mode is a mode in which the encoded data of the first quantized transform coefficient associated with the first chroma component among the plurality of chroma components is omitted in the encoded data, and the first chroma component and the second chroma component among the plurality of chroma components are integrated, wherein, in the derivation in the first mode, a first prediction error of the first chrominance component is derived using quantized transform coefficients associated with a chrominance component different from the first chrominance component, wherein, in the derivation in the first mode, the quantized transform coefficients used to derive the first prediction error of the first chrominance component are inversely quantized using a quantization matrix corresponding to a chrominance component different from the first chrominance component, The second mode is a mode of the encoded data in which the first quantized transform coefficient is not omitted in the encoded data, and In which, the quantization matrix used to inverse quantize the quantized transform coefficients used to derive the first prediction error when using the first mode is the same as the quantization matrix used to inverse quantize the quantized transform coefficients associated with chrominance components different from the first chrominance components when using the second mode.

Images