Decoding method and program

Abstract

To provide an encoding device, a decoding device, and a program that improve image quality and encoding efficiency by using an appropriate quantization matrix when a joint encoding mode is applied.SOLUTION: The encoding device 1 includes a predictor 170 configured to generate, for each component, a prediction block corresponding to an encoding-target block, a residual generator 110 configured to generate, for each component, a prediction residual representing a difference between the encoding-target block and the prediction block, and a controller 180 including a mode selector 181 configured to select one of an individual coding mode in which a transform process and a quantization process are performed on a first component prediction residual and a second component prediction residual for each single component and a joint coding mode in which a transform process and a quantization process are performed on a joint prediction residual generated from the first component prediction residual and the second component prediction residual, and a quantization controller 182 configured to determine a quantization matrix to be applied in the quantization process based on the mode selected by the mode selector.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an encoding device, a decoding device, and a program. [Background technology]

[0002] In video coding methods such as HEVC and VVC, a coding device outputs a bitstream by performing conversion processing, quantization processing, and entropy coding processing on a prediction residual, which is the difference between a predicted image generated by inter-prediction or intra-prediction and an original image.

[0003] In the quantization process, quantization is performed using a quantization parameter (QP) that is set for each block to be coded, as well as coefficients called quantization matrices that are set for each pixel in the block. Quantization matrices are introduced to control the quantization of each frequency component of the transform coefficients in order to control quality according to visual perception intensity. Quantization matrices are also sometimes called scaling lists.

[0004] The quantization matrix can be switched on a sequence or picture basis to either apply a predefined value (16) as the default value or apply a table set by the user for each block size, color component, and prediction mode.

[0005] Incidentally, in VVC, joint coding of chroma residual (JCCR) is adopted as a coefficient coding mode for color difference components (see, for example, Non-Patent Document 1).

[0006] In this joint coding mode, the encoder generates a joint prediction residual from the prediction residual of the first chrominance component (Cb component) and the prediction residual of the second chrominance component (Cr component) by utilizing the correlation between the prediction residuals of the first chrominance component (Cb component) and the second chrominance component (Cr component). For example, the encoder generates a joint prediction residual by combining the prediction residual of the second chrominance component, whose positive and negative polarities are inverted, with the prediction residual of the first chrominance component. The encoder then performs transform processing, quantization processing, and entropy coding processing on the generated joint prediction residual before transmitting it.

[0007] The decoding device reconstructs the prediction residual for the first chrominance component and the prediction residual for the second chrominance component from the transmitted joint prediction residual. In this way, by transmitting only one joint prediction residual for the two chrominance components, coding efficiency is improved.

[0008] In mode selection on the encoding device side, a rate-distortion cost is calculated from the prediction mode, the amount of information required to transmit the prediction residual, and the image quality for both the case where the joint encoding mode is applied and the case where it is not applied, and the calculated rate-distortion costs are compared to determine whether or not to apply the joint encoding mode.

[0009] When the joint coding mode is not applied, a prediction mode is likely to be selected in which the energy of the prediction residuals of the first and second chrominance components is concentrated in the low-frequency components by the transform processing applied to the prediction residuals of the first and second chrominance components, and as a result, the energy distribution of the transform coefficients of the first and second chrominance components tends to be biased toward the low-frequency components.

[0010] On the other hand, when the joint coding mode is applied, a prediction mode that reverses the sign of the prediction residuals of the first and second chrominance components tends to be selected. As a result, the energy distribution of the transform coefficients when the transform process is applied to the joint prediction residuals is likely to be different from that when the joint coding mode is not applied.

[0011] In VVC and HEVC, controlling quantization using only QP, which is a single parameter for each block, is insufficient. Therefore, in order to independently control the quantization process of the luma component and each chrominance component, it is possible to set a fixed QP offset (pps_cb_qp_offset and pps_cr_qp_offset) for each chrominance component on a picture-by-picture basis.

[0012] Furthermore, as described above, the tendency of the transform coefficients of the joint prediction residual differs depending on whether or not the joint coding mode is applied. Therefore, in the VVC committee draft, in order to control quantization independently depending on whether or not the joint coding mode is applied, it is possible to set the QP offset (pps_joint_cbcr_qp_offset) for the joint prediction residual of a block to which the joint coding mode is applied on a picture-by-picture basis. [Prior art documents] [Non-patent literature]

[0013] [Non-Patent Document 1] JVET-O2002,“Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)” DISCLOSURE OF THE INVENTION

[0014] However, in conventional methods, although the quantization matrix can be set for each component (Y, Cb, Cr), each block size, and each prediction mode, the quantization matrix for the first chrominance component is used regardless of whether the joint coding mode is applied or not.

[0015] In other words, in the conventional method, even when the joint coding mode is applied, a quantization matrix set to improve the quality of a block to which the joint coding mode is not applied is used.

[0016] In this way, using a quantization matrix set to improve the quality of blocks to which the joint coding mode is not applied for blocks to which the joint coding mode is applied may actually result in visual degradation.

[0017] Therefore, an object of the present invention is to provide an encoding device, a decoding device, and a program that improve image quality and encoding efficiency by enabling the use of an appropriate quantization matrix even when a joint encoding mode is applied.

[0018] a mode selection unit that selects one of an individual coding mode in which a transform process and a quantization process are performed on the prediction residual of the first component and the prediction residual of the second component for each individual component, and a joint coding mode in which a transform process and a quantization process are performed on a joint prediction residual generated from the prediction residual of the first component and the prediction residual of the second component; a quantization control unit that determines a quantization matrix to be used for the quantization process based on the mode selected by the mode selection unit; and a joint transform / quantization unit that, when the mode selection unit selects the joint coding mode, performs the quantization process on a joint transform coefficient generated by the transform process of the joint prediction residual using the quantization matrix determined by the quantization control unit.

[0019] A decoding device according to a second aspect is a decoding device that decodes an image composed of two or more components including a first component and a second component by dividing the image into blocks, and includes an entropy decoding unit that outputs a first flag indicating whether a joint coding mode that performs a transform process and a quantization process on a joint prediction residual generated from a prediction residual of the first component and a prediction residual of the second component has been applied on the coding device side by decoding the coded stream, and a quantized transform coefficient; an inverse quantization control unit that determines a quantization matrix to be used for an inverse quantization process on the quantized transform coefficient based on the first flag; and a joint inverse quantization unit that performs the inverse quantization process on the quantized transform coefficient output by the entropy decoding unit using the quantization matrix determined by the inverse quantization control unit when the first flag indicates that the joint coding mode has been applied.

[0020] A program according to a third aspect causes a computer to function as the encoding device according to the first aspect.

[0021] A program according to a fourth aspect causes a computer to function as the decryption device according to the second aspect.

[0022] According to the present invention, it is possible to provide an encoding device, a decoding device, and a program that improve image quality and encoding efficiency by making it possible to use an appropriate quantization matrix even when a joint encoding mode is applied. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Figure 2] FIG. 10 is a diagram illustrating an example of a quantization matrix. [Figure 3] FIG. 2 is a diagram illustrating the operation of a control unit of the encoding device according to the embodiment. [Figure 4] FIG. 1 is a diagram illustrating a configuration of a decoding device according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating the operation of a control unit of the decoding device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0024] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. The encoding device and the decoding device according to the embodiment encode and decode moving images, respectively, as typified by MPEG. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0025] <Encoding device> First, the configuration of an encoding device according to this embodiment will be described. Fig. 1 shows the configuration of an encoding device 1 according to this embodiment. The encoding device 1 is a device that performs encoding in units of blocks generated by dividing an image made up of two or more components.

[0026] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a residual generation unit 110, a transform / quantization unit 120, an entropy encoding unit 130, an inverse quantization / inverse transform unit 140, a synthesis unit 150, a memory unit 160, a prediction unit 170, and a control unit 180.

[0027] The block division unit 100 divides an input image, which is an input image in units of frames (or pictures) constituting a moving image, into a plurality of image blocks, and outputs the image blocks generated by the division to the residual generation unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, but may also be a rectangle (non-square). An image block is a unit (block to be coded) for coding by the encoding device 1, and is a unit (block to be decoded) for decoding by the decoding device. Such an image block is sometimes called a CU (Coding Unit).

[0028] The block division unit 100 divides the luminance component (Y component) and the chrominance components into blocks. The chrominance components include a first chrominance component (Cb component) and a second chrominance component (Cr component). The following mainly describes a case where the shape of the block division is the same for the luminance component and the chrominance components, but it is also possible to control the block division independently for the luminance component and the chrominance component. When there is no particular distinction between the luminance block and the chrominance block, they are simply referred to as the block to be coded. Furthermore, when there is no particular distinction between the first chrominance component and the second chrominance component, they are simply referred to as the chrominance component.

[0029] The residual generation unit 110 calculates a prediction residual representing a difference (error) between the current block to be coded output from the block division unit 100 and a prediction block generated by the prediction unit 170 predicting the current block to be coded, for each component, and outputs the calculated prediction residual to the transform and quantization unit 120. Specifically, the residual generation unit 110 outputs a prediction residual of the luminance component, a prediction residual of the first chrominance component, and a prediction residual of the second chrominance component for one current block to the transform and quantization unit 120.

[0030] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis on the prediction residual of each component output from the residual generation unit 110. The transform / quantization unit 120 includes a switching unit 121, a single-component transform / quantization unit 122, and a joint transform / quantization unit 123.

[0031] Under the control of the control unit 180, the switching unit 121 outputs the prediction residual of each component output from the residual generation unit 110 to either the single component transform and quantization unit 122 or the joint transform and quantization unit 123. The switching unit 121 outputs the prediction residual of the luminance component to the single component transform and quantization unit 122.

[0032] For a block for which an individual coding mode has been selected in which transformation processing and quantization processing are performed on the prediction residuals of the first chrominance component and the second chrominance component for each single component, the switching unit 121 outputs the prediction residuals of the first chrominance component and the second chrominance component to the single component transformation and quantization unit 122.

[0033] On the other hand, for a block for which a joint coding mode has been selected in which a transform process and a quantization process are performed on a joint prediction residual generated from the prediction residual of the first chrominance component and the prediction residual of the second chrominance component, the switching unit 121 outputs the prediction residual of the first chrominance component and the prediction residual of the second chrominance component to the joint transform / quantization unit 123.

[0034] The single component transform / quantization unit 122 performs transform processing and quantization processing on a block-by-block basis on the prediction residual of each component output from the residual generation unit 110. The single component transform / quantization unit 122 includes a single component transform unit 122a and a single component quantization unit 122b.

[0035] The single component transform unit 122a performs a transform process on the prediction residual of each component output from the residual generation unit 110 to calculate a transform coefficient for each frequency component, and outputs the calculated transform coefficient to the single component quantization unit 122b. The transform process refers to a process of converting a pixel domain signal into a frequency domain signal, such as a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), or a transform obtained by integerizing these. The transform process may also include a transform skip that adjusts the pixel domain signal by scaling or the like without converting it into a frequency domain signal.

[0036] The single component quantization unit 122b quantizes the transform coefficients of each component output from the single component transform unit 122a using a quantization parameter (QP) and a quantization matrix (scaling list), and outputs the quantized transform coefficients of each component to the entropy coding unit 130 and the inverse quantization / inverse transform unit 140. Here, the quantization parameter (QP) is a parameter that is commonly applied to all transform coefficients in a block and determines the coarseness of quantization. The quantization matrix is ​​a matrix whose elements are quantization values ​​that control the quantization step for each frequency component when quantizing each transform coefficient.

[0037] Fig. 2 is a diagram showing an example of a quantization matrix. A quantization matrix is ​​a matrix consisting of values ​​set for each component in one block. Specifically, a quantization matrix is ​​a matrix consisting of values ​​(weighting coefficients) set for each component of i x j elements according to the block size, and is used to adjust the coarseness of quantization for each component ranging from low frequency to high frequency of the transform coefficients. Fig. 2 shows an example where i x j = 4 x 4.

[0038] In the example shown in Figure 2, the quantization matrix has larger values ​​as the horizontal and vertical orders increase. In such a quantization matrix, transform coefficients associated with elements arranged further to the bottom right, i.e., transform coefficients in higher frequencies, are quantized with lower precision. Therefore, by taking advantage of the human visual characteristic that the lower frequencies are more sensitive to spatial changes in shade and hue, it is possible to reduce the amount of information in higher frequencies through quantization without degrading subjective image quality.

[0039] The quantization matrix can be set for each combination of the prediction mode (intra prediction or inter prediction) of the block to be coded, the block size (e.g., 2x2, 4x4, 8x8, 16x16, 32x32, 64x64), and one luminance and two color difference signals.

[0040] In the quantization process on the first chrominance component and the second chrominance component, the single component quantization unit 122b may perform quantization using a quantization parameter (QP) that differs for each component and a quantization matrix that stores different elements for each component. Furthermore, the quantization matrix may use a default value that is predefined by the system, or a value determined by the encoding device 1. When using the quantization parameter (QP) and quantization matrix determined by the encoding device 1, the single component quantization unit 122b outputs the quantization parameter (QP) and quantization matrix used for quantization to the entropy encoding unit 130 and the inverse quantization and inverse transform unit 140.

[0041] Meanwhile, the joint transform / quantization unit 123 generates one joint prediction residual using the correlation between the prediction residual of the first chrominance component and the prediction residual of the second chrominance component output from the residual generation unit 110, and performs a transform process and a quantization process on the generated joint prediction residual to generate a quantized joint transform coefficient, which is output to the entropy coding unit 130. In this way, by transmitting only one joint prediction residual for two chrominance components, coding efficiency is improved. The joint transform / quantization unit 123 includes a joint residual generation unit 123a, a joint transform unit 123b, and a joint quantization unit 123c.

[0042] The joint residual generation unit 123a generates a joint prediction residual from the prediction residual of the first chrominance component and the prediction residual of the second chrominance component output from the residual generation unit 110, and outputs the generated joint prediction residual to the joint conversion unit 123b. Specifically, when the first chrominance component of the prediction residual is resCb and the second chrominance component of the prediction residual is resCr, the joint residual generation unit 123a generates a joint prediction residual resJointC by the following equation (1):

[0043] resJointC[ x ][ y ] = ( resCb[ x ][ y ] + CSign * resCr[ x ][ y ] ) / 2 …(1)

[0044] Here, x and y are pixel positions within the block to be coded, and CSign is a positive or negative sign (-1 or +1) that can be set in slice units.

[0045] Furthermore, when generating the joint prediction residual, the joint residual generation unit 123a may be configured to select an optimal joint coding mode from among a plurality of joint coding modes on the encoding device side. For example, when the joint coding mode is "1", the joint residual generation unit 123a may select resJointC[ x ][ y ] = ( 4 * resCb[ x ][ y ] + 2 * CSign * resCr[ x ][ y ] ) / 5 …(2) If the joint coding mode is "2", resJointC[ x ][ y ] = ( resCb[ x ][ y ] + CSign * resCr[ x ][ y ] ) / 2 …(3) If the joint coding mode is "3", resJointC[ x ][ y ] = ( 4 * resCr[ x ][ y ] + 2 * CSign * resCb[ x ][ y ] ) / 5 …(4) In this way, a plurality of joint residual generation processes may be switched according to the joint coding mode determined on the coding device 1 side to generate joint prediction residuals.

[0046] The joint transform unit 123b performs a transform process on the joint prediction residual output from the joint residual generation unit 123a to calculate joint transform coefficients for each frequency component, and outputs the calculated joint transform coefficients to the joint quantization unit 123c.

[0047] The joint quantization unit 123c quantizes the joint transform coefficients output from the joint transform unit 123b using a quantization parameter (QP) and a quantization matrix, and outputs the quantized joint transform coefficients, which are the quantized joint transform coefficients, to the entropy coding unit 130 and the inverse quantization / inverse transform unit 140.

[0048] In this embodiment, the joint quantization unit 123c can set a quantization parameter (QP) and a quantization matrix to be applied in the quantization process of the joint transform coefficients to be different from the quantization parameters (QP) and the quantization matrix that the single component quantization unit 122b applies to the transform coefficients of the first and second chrominance components. Note that, in the quantization process of the joint transform coefficients, the joint quantization unit 123c may use a quantization matrix that stores default values ​​that are predefined by the system, or may use a quantization matrix that stores each element that is set on the encoding device 1 side for quantization of the joint transform coefficients.

[0049] The entropy coding unit 130 performs entropy coding on the quantized transform coefficients output from the transform / quantization unit 120, compresses the data, generates a coded stream (bit stream), and outputs the coded stream to the outside of the coding device 1. For the entropy coding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used.

[0050] Specifically, for a block for which the individual coding mode is selected, the prediction residual of each component is subjected to a transform process and a quantization process by the single component transform / quantization unit 122, and the generated quantized transform coefficients for each component are input to the entropy coding unit 130. The entropy coding unit 130 performs entropy coding on the input quantized transform coefficients of each component, and outputs a coded stream.

[0051] On the other hand, for a block for which the joint coding mode is selected, the joint transformation and quantization unit 123 performs transformation and quantization on the joint prediction residuals generated, and the generated quantized joint transform coefficients are input to the entropy coding unit 130. The entropy coding unit 130 performs entropy coding on the input quantized joint transform coefficients and outputs a coded stream. At this time, the single component transformation and quantization unit 122 does not perform any processing related to the prediction residuals of the first chrominance component and the second chrominance component, and the prediction residuals are not input to the entropy coding unit 130.

[0052] In addition, the entropy coding unit 130 encodes information such as the size and shape of each block to be coded input from the block division unit 100, a flag (first flag) input from the control unit 180 indicating whether or not to apply a joint coding mode, and information regarding prediction input from the prediction unit 170 (e.g., information regarding the prediction mode and motion vector).

[0053] Furthermore, when the single component quantization unit 122b and the joint quantization unit 123c perform quantization processing using a quantization parameter (QP) and a quantization matrix set on the encoding device 1 side, the entropy encoding unit 130 encodes the applied quantization parameter (QP) and quantization matrix input from the single component quantization unit 122b and the joint quantization unit 123c.

[0054] The inverse quantization and inverse transform unit 140 performs inverse quantization processing and inverse transform processing on the quantized transform coefficients output from the transform and quantization unit 120, and outputs the reconstructed prediction residuals to the synthesis unit 150. The inverse quantization and inverse transform unit 140 includes a single component inverse quantization and inverse transform unit 141 and a joint inverse quantization and inverse transform unit 142.

[0055] The single component inverse quantization and inverse transform unit 141 performs inverse quantization processing and inverse transform processing on each of the transform coefficients of each component output from the single component transform and quantization unit 122 to restore the prediction residual of each component, and outputs the restored prediction residual to the synthesis unit 150. The single component inverse quantization and inverse transform unit 141 has a single component inverse quantization unit 141a and a single component inverse transform unit 141b.

[0056] The single component inverse quantization unit 141a performs inverse quantization processing corresponding to the quantization processing performed by the single component quantization unit 122b. Specifically, the single component inverse quantization unit 141a performs inverse quantization processing using the quantized transform coefficients, quantization parameters (QPs), and quantization matrices of each component output from the single component quantization unit 122b to restore transform coefficients, and outputs the restored transform coefficients to the single component inverse transform unit 141b.

[0057] The single component inverse transform unit 141b performs inverse transform processing corresponding to the transform processing performed by the single component transform unit 122a. For example, if the single component transform unit 122a performs DCT, the single component inverse transform unit 141b performs inverse DCT. The single component inverse transform unit 141b performs inverse transform processing on the transform coefficients of each component output from the single component inverse quantization unit 141a to restore the prediction residuals of each component, and outputs the restored prediction residuals to the synthesis unit 150.

[0058] On the other hand, for a block for which the joint coding mode is selected, the joint inverse quantization and inverse transform unit 142 performs inverse quantization processing and inverse transform processing on the quantized joint transform coefficients output from the joint transform and quantization unit 123 to restore a joint prediction residual, reconstructs a prediction residual of the first chrominance component and a prediction residual of the second chrominance component from the joint prediction residual, and outputs these prediction residuals to the synthesis unit 150. The joint inverse quantization and inverse transform unit 142 includes a joint inverse quantization unit 142a, a joint inverse transform unit 142b, and a residual separation unit 142c.

[0059] The joint inverse quantization unit 142a performs inverse quantization processing corresponding to the quantization processing performed by the joint quantization unit 123c. Specifically, the joint inverse quantization unit 142a restores the joint transform coefficients by inverse quantizing the quantized joint transform coefficients output from the joint quantization unit 123c using a quantization parameter (QP) and a quantization matrix, and outputs the restored joint transform coefficients to the joint inverse transform unit 142b.

[0060] The joint inverse transform unit 142b performs an inverse transform process corresponding to the transform process performed by the joint transform unit 123b. For example, if the joint transform unit 123b performs a DCT, the joint inverse transform unit 142b performs an inverse DCT. The joint inverse transform unit 142b performs an inverse transform process on the joint transform coefficients output from the joint inverse quantization unit 142a to restore the joint prediction residual, and outputs the restored joint prediction residual to the residual separation unit 142c.

[0061] The residual separation unit 142c separates the joint prediction residual output from the joint inverse transform unit 142b to restore the prediction residual of the first color difference component and the prediction residual of the second color difference component, and outputs the restored prediction residuals to the synthesis unit 150.

[0062] Specifically, when the joint prediction residual is resJointC and the positive or negative sign (-1 or +1) preset by the system in units of slices is CSign, the residual separation unit 142c calculates the prediction residual resCb of the first chrominance component and the prediction residual resCr of the second chrominance component using the following equation (5):

[0063] resCb[ x ][ y ] = resJointC[ x ][ y ] resCr[ x ][ y ] = CSign * resJointC[ x ][ y ] …(5)

[0064] Here, x and y are the positions of pixels within the block to be coded.

[0065] Furthermore, when performing separation using the joint prediction residual, the residual separation unit 142c may perform a plurality of separation processes according to the joint coding mode set in the joint residual generation unit 123a.

[0066] For example, the residual separator 142c If the joint coding mode is "1", resCb[ x ][ y ] = resJointC[ x ][ y ] resCr[ x ][ y ] = ( CSign * resJointC[ x ][ y ] ) >> 1 …(6) If the joint coding mode is "2", resCb[ x ][ y ] = resJointC[ x ][ y ] resCr[ x ][ y ] = CSign * resJointC[ x ][ y ] …(7) If the joint coding mode is "3", resCb[ x ][ y ] = ( CSign * resJointC[ x ][ y ] ) >> 1 resCr[ x ][ y ] = resJointC[ x ][ y ] …(8) In this way, the prediction residual of the first chrominance component and the prediction residual of the second chrominance component may be calculated by switching among a plurality of separation processes according to the joint encoding mode determined on the encoding device 1 side.

[0067] The synthesis unit 150 synthesizes, for each component, the restored prediction residual output from the inverse quantization and inverse transform unit 140 with the prediction block output from the prediction unit 170 in pixel units. The synthesis unit 150 adds each pixel value of the restored prediction residual to each pixel value of the prediction block to restore (decode) the block to be coded, and outputs the restored decoded image (restored block) in block units to the memory unit 160.

[0068] The memory unit 160 accumulates the reconstructed blocks output from the synthesis unit 150 as decoded images in units of frames. The memory unit 160 outputs the stored decoded images to the prediction unit 170.

[0069] The prediction unit 170 performs prediction processing on a block-by-block basis to generate a prediction block for each component corresponding to the current block to be coded, and outputs the generated prediction block to the residual generation unit 110 and the synthesis unit 150. The prediction unit 170 includes an inter prediction unit 171, an intra prediction unit 172, and a switching unit 173.

[0070] The inter prediction unit 171 uses the decoded image stored in the memory unit 160 as a reference image to calculate a motion vector by a technique such as block matching, predicts the block to be coded, generates an inter prediction block, and outputs the generated inter prediction block to the switching unit 173. The inter prediction unit 171 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 171 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 130.

[0071] The intra prediction unit 172 selects an optimal intra prediction mode to apply to the current block to be coded from among multiple intra prediction modes, and predicts the current block to be coded using the selected intra prediction mode. The intra prediction unit 172 generates an intra prediction block by referring to decoded pixel values ​​adjacent to the current block to be coded in the decoded image stored in the memory unit 160, and outputs the generated intra prediction block to the switching unit 173. The intra prediction unit 172 also outputs information related to the selected intra prediction mode to the entropy coding unit 130.

[0072] The switching unit 173 switches between the inter-predicted block output from the inter-prediction unit 171 and the intra-predicted block output from the intra-prediction unit 172, and outputs either the predicted block to the residual generation unit 110 and the synthesis unit 150.

[0073] Next, the control unit 180 of the encoding device 1 according to this embodiment will be described. The control unit 180 controls the encoding process of the chrominance components. The control unit 180 has a mode selection unit 181 and a quantization control unit 182. Fig. 3 is a diagram showing the operation of the control unit 180 of the encoding device 1 according to this embodiment.

[0074] 3, in step S11, the mode selection unit 181 selects one of an individual coding mode in which a transform process and a quantization process are performed on the prediction residual of the first chrominance component and the prediction residual of the second chrominance component for each individual component, and a joint coding mode in which a transform process and a quantization process are performed on a joint prediction residual generated from the prediction residual of the first chrominance component and the prediction residual of the second chrominance component. The mode selection unit 181 may perform the selection process of step S11 for each block to be coded. The mode selection unit 181 outputs a flag (first flag) indicating whether the joint coding mode has been selected to the entropy coding unit 130.

[0075] For example, the mode selection unit 181 calculates a rate-distortion cost based on the prediction mode, the amount of information required to transmit the prediction residual, and the image quality for each of the cases where the joint coding mode is applied and where it is not applied, and determines whether to apply the joint coding mode by comparing the calculated rate-distortion costs.

[0076] When the mode selection unit 181 selects the individual coding mode (step S11: NO), it controls the switching unit 121 to output the prediction residual of the first chrominance component and the prediction residual of the second chrominance component to the single component transform / quantization unit 122. On the other hand, when the mode selection unit 181 selects the joint coding mode (step S11: YES), it controls the switching unit 121 to output the prediction residual of the first chrominance component and the prediction residual of the second chrominance component to the joint transform / quantization unit 123.

[0077] If the mode selection unit 181 selects the individual encoding mode (step S11: NO), in step S12, the quantization control unit 182 determines a quantization matrix to be applied to the quantization of the transform coefficients of the first chrominance component from among first candidates that are candidates for the quantization matrix corresponding to the first chrominance component, and determines a quantization matrix to be applied to the quantization of the transform coefficients of the second chrominance component from among second candidates that are candidates for the quantization matrix corresponding to the second chrominance component.

[0078] In the individual coding mode, a prediction mode is likely to be selected in which the energy of the prediction residuals of the first and second chrominance components is concentrated in low-frequency components by the transform process applied to the prediction residuals of the first and second chrominance components. As a result, the energy distribution of the transform coefficients of the first and second chrominance components tends to be biased toward low-frequency components. For this reason, the first and second candidates are assumed to be quantization matrices that match this energy distribution.

[0079] The quantization control unit 182 outputs the quantization matrix to be applied to the quantization of the transform coefficients of the first chrominance component and the quantization matrix to be applied to the quantization of the transform coefficients of the second chrominance component to the single component quantization unit 122b and the single component inverse quantization unit 141a.

[0080] On the other hand, if the mode selection unit 181 selects the joint coding mode (step S11: YES), in step S13, the quantization control unit 182 determines a quantization matrix to be applied to the quantization of the joint transform coefficients from among a third candidate different from the first and second candidates.

[0081] Here, the third candidate is a candidate quantization matrix prepared for quantization of the joint transform coefficients, and includes at least one of a quantization matrix storing default values ​​predefined by the system and a quantization matrix storing elements set by the encoding device 1 for quantization of the joint transform coefficients.

[0082] The quantization control unit 182 outputs the quantization matrix to be applied to the quantization of the joint transform coefficients to the joint quantization unit 123c and the joint inverse quantization unit 142a.

[0083] As described above, according to this embodiment, focusing on the fact that the energy distribution of the transform coefficients when the joint coding mode is applied is likely to be different from the energy distribution of the transform coefficients when the individual coding mode is applied, the method of determining the quantization matrix is ​​made different between the individual coding mode and the joint coding mode. Specifically, when the joint coding mode is applied, the control unit 180 determines the quantization matrix to be used for the quantization process of the joint transform coefficients from among the quantization matrix candidates (third candidates) prepared for the joint coding mode.

[0084] As a result, when the joint coding mode is applied, the coding device 1 can perform quantization processing using a quantization matrix set to improve the quality of the block to which the joint coding mode is applied. In other words, when the joint coding mode is applied, it becomes possible to use an appropriate quantization matrix, thereby improving image quality and coding efficiency.

[0085] <Decryption device> Next, a decoding device according to this embodiment will be described, focusing on differences from the above-described encoding device 1. Fig. 4 is a diagram showing the configuration of a decoding device 2 according to this embodiment. The decoding device 2 is a device that decodes a current block from an encoded stream.

[0086] As shown in FIG. 4, the decoding device 2 includes an entropy decoding unit 200 , an inverse quantization and inverse transform unit 210 , a synthesis unit 220 , a memory unit 230 , a prediction unit 240 , and a control unit 250 .

[0087] The entropy decoding unit 200 decodes the coded stream generated by the coding device 1 and decodes various types of signaling information. Specifically, the entropy decoding unit 200 obtains a flag (first flag) indicating whether a joint coding mode is applied to the block to be decoded by decoding the coded stream, and outputs the obtained flag to the control unit 250. The entropy decoding unit 200 also obtains information on the quantization process applied to the block to be decoded by decoding the coded stream, and outputs the obtained information to the inverse quantization and inverse transform unit 210. The entropy decoding unit 200 also obtains information on the prediction applied to the block to be decoded (e.g., prediction type information, motion vector information) by decoding the coded stream, and outputs the obtained information to the prediction unit 240.

[0088] Furthermore, the entropy decoding unit 200 obtains quantized transform coefficients by decoding the coded stream, and outputs the obtained quantized transform coefficients to the inverse quantization and inverse transform unit 210 .

[0089] The inverse quantization and inverse transform unit 210 performs inverse quantization and inverse transform processing on the quantized transform coefficients output from the entropy decoding unit 200, and outputs the reconstructed prediction residuals to the synthesis unit 220. The inverse quantization and inverse transform unit 210 includes a single component inverse quantization and inverse transform unit 211 and a joint inverse quantization and inverse transform unit 212.

[0090] The single component inverse quantization and inverse transform unit 211 performs inverse quantization processing and inverse transform processing on each of the transform coefficients of each component output from the entropy decoding unit 200 to restore the prediction residual of each component, and outputs the restored prediction residual to the synthesis unit 220. The single component inverse quantization and inverse transform unit 211 has a single component inverse quantization unit 211a and a single component inverse transform unit 211b.

[0091] The single component inverse quantization unit 211a performs inverse quantization processing corresponding to the quantization processing performed by the single component quantization unit 122b of the encoding device 1. Specifically, the single component inverse quantization unit 211a performs inverse quantization processing using a quantization parameter (QP) and a quantization matrix to restore transform coefficients, and outputs the restored transform coefficients to the single component inverse transform unit 211b.

[0092] The single component inverse transform unit 211b performs inverse transform processing corresponding to the transform processing performed by the single component transform unit 122a of the encoding device 1. For example, if the single component transform unit 122a performs DCT, the single component inverse transform unit 211b performs inverse DCT. The single component inverse transform unit 211b performs inverse transform processing on the transform coefficients of each component output from the single component inverse quantization unit 211a to restore the prediction residuals of each component, and outputs the restored prediction residuals to the synthesis unit 220.

[0093] For a block for which the joint coding mode is selected, the single component inverse quantization and inverse transform unit 211 does not process the first chrominance component and the second chrominance component, and does not output them to the synthesis unit 220 either.

[0094] On the other hand, for a block for which the joint coding mode is selected, the joint inverse quantization and inverse transform unit 212 performs inverse quantization processing and inverse transform processing on the quantized joint transform coefficients output from the entropy decoding unit 200 to restore a joint prediction residual, reconstructs a prediction residual of the first chrominance component and a prediction residual of the second chrominance component from the joint prediction residual, and outputs these prediction residuals to the synthesis unit 220. The joint inverse quantization and inverse transform unit 212 includes a joint inverse quantization unit 212a, a joint inverse transform unit 212b, and a residual separation unit 212c.

[0095] The joint inverse quantization unit 212a performs inverse quantization processing corresponding to the quantization processing performed by the joint quantization unit 123c of the encoding device 1. Specifically, the joint inverse quantization unit 212a restores the joint transform coefficients by inverse quantizing the quantized joint transform coefficients output from the entropy decoding unit 200 using a quantization parameter (QP) and a quantization matrix, and outputs the restored joint transform coefficients to the joint inverse transform unit 212b.

[0096] In this embodiment, the joint inverse quantization unit 212a allows the quantization parameter (QP) and quantization matrix applied in the inverse quantization process of the quantized joint transform coefficients to be set differently from the quantization parameter (QP) and quantization matrix applied by the single component inverse quantization unit 211a to the quantized transform coefficients of the first chrominance component and the second chrominance component. Note that, in the inverse quantization process of the quantized joint transform coefficients, the joint inverse quantization unit 212a may use a quantization matrix in which default values ​​defined in advance by the system are stored, or may use a quantization matrix in which each element set on the encoding device 1 side is stored.

[0097] The joint inverse transform unit 212b performs inverse transform processing corresponding to the transform processing performed by the joint transform unit 123b of the encoding device 1. For example, if the joint transform unit 123b performs DCT, the joint inverse transform unit 212b performs inverse DCT. The joint inverse transform unit 212b performs inverse transform processing on the joint transform coefficients output from the joint inverse quantization unit 212a to restore joint prediction residuals, and outputs the restored joint prediction residuals to the residual separation unit 212c.

[0098] It should be noted that the composite inverse quantization and inverse transform unit 212 does not perform processing on blocks for which the individual coding mode is selected.

[0099] The residual separation unit 212c separates the joint prediction residual output from the joint inverse transform unit 212b to restore the prediction residual of the first chrominance component and the prediction residual of the second chrominance component, and outputs the restored prediction residuals to the synthesis unit 220.

[0100] Specifically, the residual separation unit 212c calculates the prediction residual resCb of the first chrominance component and the prediction residual resCr of the second chrominance component using the above equation (5). The residual separation unit 212c may perform separation processing using any of the above equations (6) to (8), depending on the joint coding mode selected by the coding device 1.

[0101] The synthesis unit 220 synthesizes, for each component, the restored prediction residual output from the inverse quantization and inverse transform unit 210 with the prediction block output from the prediction unit 240 in pixel units. The synthesis unit 220 adds each pixel value of the restored prediction residual to each pixel value of the prediction block to restore (decode) the block to be decoded, and outputs the restored decoded image (restored block) in block units to the memory unit 230.

[0102] The memory unit 230 accumulates the reconstructed blocks output from the synthesis unit 220 as decoded images in units of frames. The memory unit 230 outputs the stored decoded images to the prediction unit 240.

[0103] The prediction unit 240 performs prediction processing on a block-by-block basis to generate a prediction block for each component corresponding to the block to be decoded, and outputs the generated prediction block to the residual generation unit 110 and the synthesis unit 220. The prediction unit 240 includes an inter prediction unit 241, an intra prediction unit 242, and a switching unit 243.

[0104] Based on the information output from the entropy decoding unit 200, the inter prediction unit 241 predicts the block to be decoded using the decoded image stored in the memory unit 230 as a reference image to generate an inter prediction block, and outputs the generated inter prediction block to the switching unit 243.

[0105] Based on the information output from the entropy decoding unit 200, the intra prediction unit 242 predicts the block to be decoded using an intra prediction mode selected from multiple intra prediction modes to generate an intra prediction block, and outputs the generated intra prediction block to the switching unit 243.

[0106] The switching unit 243 switches between the inter-predicted block output from the inter-prediction unit 241 and the intra-predicted block output from the intra-prediction unit 242, and outputs either the predicted block to the residual generation unit 110 and the synthesis unit 220.

[0107] Next, the control unit 250 of the decoding device 2 according to this embodiment will be described. The control unit 250 controls the decoding process of the chrominance components. The control unit 250 has an inverse quantization control unit 251. Fig. 5 is a diagram showing the operation of the control unit 250 of the decoding device 2 according to this embodiment.

[0108] 5, in step S21, the inverse quantization control unit 251 identifies either the individual coding mode or the joint coding mode based on a flag (first flag) output from the entropy decoding unit 200. The inverse quantization control unit 251 may perform the process of step S21 for each block to be decoded.

[0109] If the flag indicates the individual encoding mode (step S21: NO), in step S22, the inverse quantization control unit 251 determines a quantization matrix to be applied to the inverse quantization of the quantized transform coefficients of the first chrominance component from among first candidates that are candidates for the quantization matrix corresponding to the first chrominance component, and determines a quantization matrix to be applied to the inverse quantization of the quantized transform coefficients of the second chrominance component from among second candidates that are candidates for the quantization matrix corresponding to the second chrominance component.

[0110] In the individual coding mode, a prediction mode is likely to be selected in which the energy of the prediction residuals of the first and second chrominance components is concentrated in low-frequency components by the transform process applied to the prediction residuals of the first and second chrominance components. As a result, the energy distribution of the transform coefficients of the first and second chrominance components tends to be biased toward low-frequency components. For this reason, the first and second candidates are assumed to be quantization matrices that match this energy distribution.

[0111] The inverse quantization control unit 251 outputs the quantization matrix to be applied to the inverse quantization of the quantized transform coefficients of the first chrominance component and the quantization matrix to be applied to the inverse quantization of the quantized transform coefficients of the second chrominance component to the single component inverse quantization unit 211a.

[0112] On the other hand, if the flag indicates a composite coding mode (step S21: YES), in step S23, the inverse quantization control unit 251 determines a quantization matrix to be applied to the inverse quantization of the quantized joint transform coefficients from among a third candidate different from the first and second candidates.

[0113] Here, the third candidate is a candidate quantization matrix prepared for inverse quantization of the quantized joint transform coefficients. The third candidate includes at least one of a quantization matrix storing default values ​​predefined by the system and a quantization matrix storing elements set by the encoding device 1 for inverse quantization of the quantized joint transform coefficients.

[0114] The inverse quantization control unit 251 outputs the quantization matrix to be applied to the inverse quantization of the quantized joint transform coefficients to the joint inverse quantization unit 212a.

[0115] As described above, according to this embodiment, focusing on the fact that the energy distribution of the transform coefficients when the joint coding mode is applied is likely to be different from the energy distribution of the transform coefficients when the individual coding mode is applied, the method of determining the quantization matrix is ​​made different between the individual coding mode and the joint coding mode. Specifically, for a block to which the joint coding mode is applied, the control unit 250 determines a quantization matrix to be used in the quantization process of the joint transform coefficients from among the quantization matrix candidates (third candidates) prepared for the joint coding mode.

[0116] As a result, when the joint coding mode is applied, the decoding device 2 can perform inverse quantization processing using a quantization matrix set to improve the quality of the block to which the joint coding mode is applied. In other words, when the joint coding mode is applied, it becomes possible to use an appropriate quantization matrix, thereby improving image quality and coding efficiency.

[0117] <Example of change> Next, a modified example of the embodiment will be described, focusing mainly on the differences from the above-described embodiment.

[0118] In the above-described embodiment, the quantization control unit 182 of the encoding device 1 and the inverse quantization control unit 251 of the decoding device 2 determine the quantization matrix to be used in the quantization process based on whether or not the joint coding mode is applied.

[0119] On the other hand, the quantization control unit 182 of the encoding device 1 and the inverse quantization control unit 251 of the decoding device 2 in this modified example determine the quantization matrix to be used for the quantization process based on whether the joint encoding mode is applied or not and a flag (second flag) indicating the quantization matrix to be used when the joint encoding mode is applied.

[0120] Specifically, the quantization control unit 182 of the encoding device 1 determines either the quantization matrix that the single component quantization unit 122b applies to the transform coefficients of the first chrominance component (hereinafter referred to as the "first chrominance quantization matrix") or the quantization matrix that the single component quantization unit 122b applies to the transform coefficients of the second chrominance component (hereinafter referred to as the "second chrominance quantization matrix") as the quantization matrix to be used in the quantization process for the joint transform coefficients output by the joint transform unit 123b (hereinafter referred to as the "chrominance joint quantization matrix").

[0121] When the quantization control unit 182 determines the first chrominance quantization matrix as the chrominance joint quantization matrix, the joint quantization unit 123c of the encoding device 1 performs quantization processing on the joint transform coefficients output by the joint transform unit 123b using the quantization parameter (QP) set for the block to be encoded and the first chrominance quantization matrix.

[0122] On the other hand, when the quantization control unit 182 determines the second chrominance quantization matrix as the chrominance joint quantization matrix, the joint quantization unit 123c of the encoding device 1 performs quantization processing on the joint transform coefficients output by the joint transform unit 123b using the quantization parameter (QP) set for the block to be encoded and the second chrominance quantization matrix.

[0123] The entropy coding unit 130 of the coding device 1 outputs a bitstream including a first flag indicating whether to apply a joint coding mode and a second flag indicating a quantization matrix (chrominance joint quantization matrix) to be used when the joint coding mode is applied. The second flag is a flag (1-bit information) indicating whether to use the first chrominance quantization matrix or the second chrominance quantization matrix as the chrominance joint quantization matrix.

[0124] The entropy coding unit 130 signals (i.e., transmits to the decoding side) the second flag in units of a sequence, a picture, or a slice. For example, the entropy coding unit 130 may signal the second flag in a Sequence Parameter Set (SPS), which is a set of coding information for a sequence, and control the same quantization matrix for all pictures / slices included in the sequence that refer to the SPS.

[0125] Alternatively, the entropy coding unit 130 may signal the second flag in a Picture Parameter Set (PPS), which is a set of coding information for a picture, and control the selection of a quantization matrix in a picture that refers to the PPS.

[0126] Alternatively, the entropy coding unit 130 may signal a second flag in a Picture Header (PH) containing coding information for each picture to be coded or in a Slice Header (SH) containing coding information for each slice, and switch the selection control of the quantization matrix for each picture / slice.

[0127] Alternatively, the entropy coding unit 130 may signal the second flag in an Adaptation Parameter Set (APS) that includes information about the scaling list.

[0128] In the decoding device 2 according to this modified example, the entropy decoding unit 200 decodes the coded stream generated by the coding device 1, and outputs a first flag indicating whether the joint coding mode is applied to the block to be decoded, and a second flag indicating whether the chroma joint quantization matrix used by the joint inverse quantization unit 212a is the first chroma quantization matrix or the second chroma quantization matrix. The second flag is a flag indicating the chroma joint quantization matrix to be used when the joint coding mode is applied, on a sequence-by-sequence, picture-by-picture, or slice-by-slice basis.

[0129] The inverse quantization control unit 251 of the decoding device 2 determines a chroma joint quantization matrix to be used by the joint inverse quantization unit 212a based on the first flag and the second flag. Specifically, when the first flag indicates that the joint coding mode is applied to the block to be decoded, the inverse quantization control unit 251 determines the first chroma quantization matrix or the second chroma quantization matrix indicated by the second flag as the chroma joint quantization matrix.

[0130] The joint inverse quantization unit 212a performs inverse quantization processing corresponding to the quantization processing performed by the joint quantization unit 123c of the encoding device 1. Specifically, the joint inverse quantization unit 212a restores the joint transform coefficients by inverse quantizing the quantized joint transform coefficients output from the entropy decoding unit 200 using a quantization parameter (QP) and a chrominance joint quantization matrix determined by the inverse quantization control unit 251, and outputs the restored joint transform coefficients to the joint inverse transform unit 212b.

[0131] In this way, according to this modified example, by determining the chroma joint quantization matrix to be used for the quantization process based on the first flag indicating whether the joint coding mode is applied or not and the second flag set for each sequence, each picture, each slice, etc., it is possible to realize flexible quantization process without transmitting a new quantization matrix for the joint coding mode from the encoding side to the decoding side.

[0132] In this modified example, an example has been described in which the second flag is a flag indicating either the first chroma quantization matrix or the second chroma quantization matrix.

[0133] However, the second flag may be a flag indicating any one of the first chroma quantization matrix, the second chroma quantization matrix, and the luma quantization matrix. The luma quantization matrix refers to the quantization matrix that the single component quantization unit 122b applies to the transform coefficients of the luma component. In this case, the second flag may be 2-bit information.

[0134] The second flag may be a flag indicating any one of the first chroma quantization matrix, the second chroma quantization matrix, the luma quantization matrix, and another quantization matrix (i.e., a new quantization matrix for the joint coding mode). The entropy coding unit 130 may signal the new quantization matrix to the decoding side only if the second flag indicates a new quantization matrix for the joint coding mode. If the second flag indicates a new quantization matrix for the joint coding mode, the entropy decoding unit 200 obtains the new quantization matrix signaled from the coding side.

[0135] <Other embodiments> In the above-described embodiment, an example has been described in which the first component is the first chrominance component Cb and the second component is the second chrominance component Cr. However, the present invention is applicable to cases in which an image composed of two or more components is encoded and decoded. For example, the operation according to the above-described embodiment may be applied to an image composed of an R component, a G component, and a B component.

[0136] A program may be provided that causes a computer to execute each process performed by the encoding device 1. A program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded on a computer-readable medium. Using a computer-readable medium makes it possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0137] The circuits that execute the processes performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits that execute the processes performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0138] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0139] This application claims priority from Japanese Patent Application No. 2019-172329 (filed September 20, 2019), the entire contents of which are incorporated herein by reference.

Claims

1. A decoding method for dividing an image made up of two or more components including a first component and a second component and decoding each of the generated blocks, outputting, on a block-by-block basis, a first flag indicating whether a joint coding mode for performing a transform process and a quantization process on a joint prediction residual generated from the prediction residual of the first component and the prediction residual of the second component has been applied on an encoding device side, a 1-bit second flag used to control an inverse quantization process when the joint coding mode has been applied, and a quantized transformation coefficient, based on the encoded stream; determining a quantization matrix to be used in an inverse quantization process on the quantized transform coefficients based on the first flag; performing the inverse quantization process on the quantized transform coefficients using the determined quantization matrix when the first flag indicates that the joint coding mode has been applied; a decoding method for controlling the inverse quantization process based on the second flag only when the first flag indicates that the joint coding mode has been applied;

2. the second flag indicates a quantization matrix to be used in a picture unit when the joint coding mode is applied; 2. The decoding method according to claim 1, wherein the quantization matrix to be used in the inverse quantization process is determined based on the first flag and the second flag.

3. 3. The decoding method according to claim 2, wherein the second flag indicates either a quantization matrix to be applied to the first component or a quantization matrix to be applied to the second component as a quantization matrix to be used when the joint encoding mode is applied.

4. 4. The decoding method according to claim 1, wherein the first component is a first chrominance component and the second component is a second chrominance component.

5. A program causing a computer to execute the decoding method according to claim 1.

Images