Decoding device, program, and decoding method

The decoding device enhances encoding efficiency by correctly applying chrominance residual scaling through entropy decoding and luminance-based control, addressing incorrect scaling in existing technologies.

JP2024161240AActive Publication Date: 2024-11-15NIPPON HOSO KYOKAI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2024153093
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2020-12-17
Filing Date
2024-09-05
Publication Date
2024-11-15
Estimated Expiration
2041-12-17

AI Technical Summary

Technical Problem

Existing decoding devices fail to correctly apply chrominance residual scaling, leading to a decrease in encoding efficiency due to incorrect on/off control based on significant coefficient flags.

Method used

A decoding device that performs decoding on blocks of an image divided into color components, using entropy decoding and inverse quantization to determine if color space conversion is applied, and then applies chrominance residual scaling based on luminance components and color space conversion flags, ensuring correct application of chrominance residual scaling.

Benefits of technology

Improves coding efficiency by appropriately applying color difference residual scaling, ensuring accurate transformation and reconstruction of image blocks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161240000001_ABST
    Figure 2024161240000001_ABST
Patent Text Reader

Abstract

To appropriately apply color difference residual scaling to improve coding efficiency.SOLUTION: A decoding method includes the steps of outputting a conversion coefficient for each color component of a block, a first flag indicating whether the block contains a non-zero conversion coefficient for each color component, and a second flag indicating whether the block is encoded by using a color space conversion that converts a color space of the predicted residual from a color space of the original image to another color space by decoding a bit stream, performing color space inverse conversion on the predicted residual restored from the conversion coefficient when the second flag indicates that the block is encoded using the color space conversion, and determining whether to perform color difference residual scaling in which the predicted residual of the color difference component is scaled on the basis of a luminance component corresponding to the color difference component on the basis of the first flag of the color difference component and the second flag.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a decoding device, a program, and a decoding method. [Background technology]

[0002] Non-Patent Document 1 specifies color space conversion (ACT: Adaptive Colour Transform) used for coding RGB4:4:4 video in VVC (Versatile Video Coding). Color space conversion is a technology that converts prediction residuals in the RGB color space to the YCgCo color space to remove correlation between color components of the prediction residuals and improve coding efficiency.

[0003] The encoding device performs an orthogonal transform on the prediction residual converted to the YCgCo color space for each color component (Y, Cg, Co components), quantizes and entropy codes the transform coefficients, and outputs the stream. The decoding device entropy decodes the transmitted transform coefficients, and performs an inverse color space transform (inverse ACT) on the prediction residual in the YCgCo color space obtained by inverse quantization and inverse orthogonal transform to convert it into a prediction residual in the RGB color space, and obtains a decoded image by combining the prediction residual with a predicted image.

[0004] Incidentally, VVC employs a technique called Chroma Residual Scaling (CRS), which scales the prediction residual of a chrominance component according to the corresponding luminance component.

[0005] In order to reduce the amount of calculation for chrominance residual scaling, the decoding device controls whether to apply chrominance residual scaling based on significant coefficient flags (tu_cb_coded_flag and tu_cr_coded_flag) indicating whether non-zero transform coefficients of chrominance components are transmitted. Specifically, the decoding device performs chrominance residual scaling only when the significant coefficient flags indicate that non-zero transform coefficients of chrominance components are transmitted. [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] JVET-R2001 “Versatile Video Coding Draft 9” Summary of the Invention [Problem to be solved by the invention]

[0007] When color space conversion is applied, the transform coefficients transmitted to the decoding device are in YCgCo color space, tu_cb_coded_flag is set to TRUE (“1”) if there are non-zero transform coefficients for the Cg component, and tu_cr_coded_flag is set to TRUE (“1”) if there are non-zero transform coefficients for the Co component.

[0008] If there are non-zero transform coefficients in any of the Y, Cg, or Co color components, their energy will be distributed to each color component of the RGB color space by the color space inverse transform in the decoding device, so there is a high possibility that prediction residuals will occur in all color components of the RGB color space.

[0009] However, in Non-Patent Document 1, the decoding device controls the on / off of chrominance residual scaling based on tu_cb_coded_flag and tu_cr_coded_flag. Therefore, when the significant coefficient flag of a chrominance component is FALSE ("0"), the decoding device does not perform chrominance residual scaling even if prediction residuals occur in all color components, RGB, due to color space inverse transform. This causes a problem that chrominance residual scaling is not applied correctly, resulting in a decrease in coding efficiency.

[0010] Therefore, an object of the present invention is to provide a decoding device, a program, and a decoding method that improve coding efficiency by appropriately applying chrominance residual scaling. [Means for solving the problem]

[0011] A decoding device according to a first aspect performs decoding processing on blocks obtained by dividing an original image composed of a plurality of color components, and includes an entropy decoding unit that outputs, by decoding a bitstream, transform coefficients for each color component of the block, a first flag indicating, for each color component, whether the block includes a non-zero transform coefficient, and a second flag indicating whether the block is coded using a color space transform that converts a color space of a prediction residual from the color space of the original image to another color space; an inverse quantization and inverse transform unit that restores the prediction residual from the transform coefficients for each color component; a color space inverse transform unit that performs a color space inverse transform on the prediction residual when the second flag indicates that the block is coded using the color space transform; and a scaling unit that performs chrominance residual scaling to scale the prediction residual of a chrominance component based on a luminance component corresponding to the chrominance component, and the scaling unit determines whether to perform the chrominance residual scaling based on the first flag and the second flag of the chrominance component.

[0012] The program according to the second aspect causes a computer to function as the decryption device according to the first aspect.

[0013] A decoding method according to a third aspect is a decoding method for performing a decoding process on blocks obtained by dividing an original image composed of a plurality of color components, and includes the steps of: decoding a bitstream to output transform coefficients for each color component of the block, a first flag indicating, for each color component, whether the block includes a non-zero transform coefficient, and a second flag indicating whether the block is coded using a color space transform that converts a color space of a prediction residual from the color space of the original image to another color space; performing a color space inverse transform on the prediction residual reconstructed from the transform coefficients, if the second flag indicates that the block is coded using the color space transform; and determining, based on the first flag and the second flag for a chrominance component, whether to perform chrominance residual scaling, which scales the prediction residual of the chrominance component based on a luminance component corresponding to the chrominance component. Effect of the Invention

[0014] According to the present invention, it is possible to provide a decoding device, a program, and a decoding method that improve coding efficiency by appropriately applying chrominance residual scaling. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an encoding device according to an embodiment. [Diagram 2] 11 is a graph illustrating an example of a relationship between an input pixel value and an output pixel value in a luminance mapping process according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a configuration of a decoding device according to an embodiment. [Figure 4] FIG. 11 is a diagram illustrating on / off control of chrominance residual scaling according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] An encoding device and a decoding device according to an embodiment will be described with reference to the drawings. The encoding device and the decoding device according to the embodiment respectively encode and decode moving images as represented by MPEG (Moving Picture Experts Group). In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] <Encoding device configuration> First, the configuration of the encoding device according to this embodiment will be described. Fig. 1 is a diagram showing the configuration of an encoding device 1 according to this embodiment.

[0018] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a luminance mapping unit 101, a residual generation unit 102, a scaling unit 103, a color space conversion unit 104, a transform / quantization unit 120, an entropy encoding unit 107, a dequantization / inverse transform unit 130, a color space inverse transform unit 110, a scaling unit 111, a synthesis unit 112, a luminance inverse mapping unit 113, an in-loop filter 114, a memory 115, and a prediction unit 140.

[0019] The block division unit 100 divides an original image, which is an input image in units of frames (or pictures) constituting a moving image, into a plurality of image blocks, and outputs the image blocks obtained by division to the residual generation unit 102. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, but may be a rectangle (non-square). The image block is a unit for which the encoding device 1 performs the encoding process (i.e., a block to be encoded) and a unit for which the decoding device performs the decoding process (i.e., a block to be decoded). Such an image block is sometimes called a CU (Coding Unit).

[0020] The input image may be an RGB signal with a chroma format of 4:4:4. The RGB color space is an example of a first color space. The "G" component corresponds to the first color component, the "B" component corresponds to the second color component, and the "R" component corresponds to the third color component. The block division unit 100 performs block division on each of the R, G, and B components that make up the image, thereby outputting a block for each color component. In the following description of the encoding device, when the color components are not distinguished, they are simply referred to as the block to be encoded.

[0021] The luminance mapping unit 101 performs a mapping process on each pixel value in the block to be coded of the luminance component output by the block division unit 100 based on a mapping table, thereby generating and outputting a new mapped block to be coded of the luminance component.

[0022] 2 is a graph showing an example of the relationship between input pixel values ​​and output pixel values ​​in the luminance mapping process according to the present embodiment, in which the horizontal axis represents the value of the input signal, and the vertical axis represents the value of the output signal.

[0023] As shown in Fig. 2, the mapping table is a table set for one or more slices, and is a coefficient table for expressing the relationship between an input signal before mapping processing and an output signal after mapping processing. Specifically, the mapping table stores values ​​indicating the number of pixel values ​​of the converted output signal to be assigned to each band obtained by dividing the input signal before mapping (pixel value to be mapped) from the minimum value to the maximum value into a predetermined number (N).

[0024] For example, the mapping table will be described taking the case where the number of bands N is 16 in mapping processing of a 10-bit image signal as an example. The minimum value 0 that the input signal can take before mapping, through to the maximum value 1023, are assigned as input signals corresponding to each of the equally divided bands. For example, the first band corresponds to input pixel values ​​0 through 63, while the second band corresponds to input pixel values ​​64 through 127. Input signals are assigned in a similar manner up to the 16th band.

[0025] Each band corresponds to the position of each coefficient in the mapping table. The coefficients stored in the mapping table represent the number of output pixel values ​​assigned to each band. For example, the mapping table lmcs CW ={39, 40, 55, 70, 80, 90, 97, 97, 104, 83, 57, 55, 49, 44, 34, 30}, the output pixel values ​​corresponding to the first band are 0 to 38, and the output pixel values ​​corresponding to the second band are 39 to 78. The same applies to the third through sixteenth bands. If the value corresponding to a band in the mapping table is large, the number of output pixel values ​​assigned to that band is large, and conversely, if the value is small, the number of output pixel values ​​assigned to that band is small.

[0026] The mapping table may be set by the encoding device 1 according to the frequency of occurrence of the values ​​of the luminance signal of the original image of one or more slices, may be selected by the encoding device 1 from among a plurality of mapping tables predefined by the system, or may use a mapping table predefined by the system. Note that the mapping table may store values ​​indicating the number of pixel values ​​of the input signal before conversion to be assigned to each band obtained by dividing the range from the minimum value to the maximum value that the output signal after mapping can take into a predefined number, or may quantize and hold the values ​​in the mapping table, and is not limited to the above example as long as it indicates the relationship between the input signal and the output signal before and after mapping.

[0027] Furthermore, when the encoding device 1 sets a mapping table according to the frequency of occurrence of luminance signal values ​​or selects from among multiple mapping tables, the encoding device 1 transmits the mapping table information to the decoding device 2 by some means. For example, the encoding device 1 may entropy-encode the table value information and output it as a stream. Furthermore, the encoding device 1 and the decoding device 2 may switch between mapping tables prepared in advance based on video format information (e.g., parameters indicating the relationship between optical signals and electrical signals in a video signal).

[0028] The residual generation unit 102 calculates a prediction residual representing the difference (error) between the encoding target block output by the block division unit 100 and a prediction block obtained by predicting the encoding target block by the prediction unit 140. Specifically, the residual generation unit 102 calculates a prediction residual for each color component by subtracting each pixel value of the prediction block from each pixel value of the encoding target block, and outputs the calculated prediction residual. That is, the residual generation unit 102 generates a prediction residual for each color component based on the difference between the encoding target block for each color component and the prediction block for each color component.

[0029] The scaling unit 103 performs chrominance residual scaling on the prediction residual of the chrominance component output by the residual generation unit 102. The chrominance residual scaling is a process of scaling the prediction residual of the chrominance component according to the corresponding luminance component. Note that if the luminance mapping unit 101 does not perform luminance mapping, the chrominance residual scaling is disabled.

[0030] The chrominance residual scaling depends on the average value of the decoded adjacent luminance pixel values ​​above and / or to the left of the block to be coded. The scaling unit 103 calculates an index Y Idx Calculate the scaling factor C ScaleInv cScaleInv[Y Idx ], where cScaleInv[] is a lookup table. While luminance mapping is performed for each pixel value, the scaling unit 103 performs chrominance residual scaling for the entire block to be coded for the chrominance component. Specifically, the scaling unit 103 scales the prediction residual of the chrominance component by C Res Then, the prediction residual C of the chrominance component after scaling ResScale , C Res *C Scale , i.e., C Res / C ScaleInv Calculate and output as follows.

[0031] The color space conversion unit 104 performs color space conversion on the prediction residual of each color component and outputs the prediction residual after the color space conversion. For example, the color space conversion unit 104 generates a prediction residual in the YCgCo color space by performing the following conversion calculation on the R component, G component, and B component of the prediction residual of the encoding target block.

[0032] Co=RB t=B+(Co>>1) Cg=Gt Y=t+(Cg>>1)

[0033] Here, ">>" represents a right shift operation. The "Y" component corresponds to the first color component, the "Cg" component corresponds to the second color component, and the "Co" component corresponds to the third color component. Such a YCgCo color space is an example of a second color space.

[0034] The color space conversion in the color space conversion unit 104 may generate a prediction residual composed of a new color component by performing addition, subtraction, multiplication, division, shift processing, etc. on each color component. The color space conversion does not have to be a conversion that affects all color components. For example, the color space conversion unit 104 may apply a color space conversion in which the first color component is maintained unchanged, the average value of the second color component and the third color component is set as the new second color component, and the difference between the second color component and the third color component is set as the new third color component.

[0035] The transform / quantization unit 120 performs transform processing and quantization processing on a block-by-block basis for each color component. The transform / quantization unit 120 includes a transform unit 105 and a quantization unit .

[0036] The transform unit 105 performs a transform process on the prediction residual (referred to as a prediction residual regardless of whether or not color space transform is applied), calculates a transform coefficient, and outputs the calculated transform coefficient. Specifically, the transform unit 105 performs a transform process on the prediction residual of each color component in units of blocks to generate a transform coefficient of each color component. The transform process may be, for example, a frequency transform such as a DCT (Discrete Cosine Transform), a DST (Discrete Sine Transform), or a discrete wavelet transform. The transform unit 105 also outputs information related to the transform process to the entropy coding unit 107.

[0037] The conversion process includes a conversion skip in which the conversion process is not performed. The conversion skip also includes a conversion in which only a horizontal conversion process is applied, or a conversion in which only a vertical conversion process is applied. In addition, the conversion unit 105 may perform a secondary conversion process in which a further conversion process is applied to the conversion coefficients obtained by the conversion process. The secondary conversion process may be applied only to a partial area of ​​the conversion coefficients.

[0038] The quantization unit 106 quantizes the transform coefficients output by the transform unit 105 using the quantization parameters and the scaling list, and outputs the quantized transform coefficients. The quantization unit 106 also outputs information related to the quantization process (specifically, information on the quantization parameters and the scaling list used in the quantization process) to the entropy coding unit 107 and the inverse quantization unit 108.

[0039] The entropy coding unit 107 performs entropy coding on the quantized transform coefficients output by the quantization unit 106, compresses the data, generates a bit stream (coded data), and outputs the bit stream to the decoding side. For the entropy coding, Huffman code, CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. can be used. In addition, the entropy coding unit 107 includes information on the conversion process from the conversion unit 105 in the bit stream and signals it to the decoding side, and includes information on the prediction process from the prediction unit 140 in the bit stream and signals it to the decoding side.

[0040] In addition, the entropy coding unit 107 includes in the bit stream a significant coefficient flag indicating whether the block to be coded includes a non-zero transform coefficient for each of the first color component (the “G” component in the RGB color space, the “Y” component in the YCgCo color space), the second color component (the “B” component in the RGB color space, the “Cg” component in the YCgCo color space), and the third color component (the “R” component in the RGB color space, the “Co” component in the YCgCo color space), and signals this to the decoding side. The significant coefficient flag is an example of a first flag.

[0041] For example, the entropy coding unit 107 sets the significant coefficient flag (tu_y_coded_flag) to TRUE ("1") if the block to be coded for the "Y" component in the YCgCo color space contains a non-zero transform coefficient, and sets the significant coefficient flag (tu_y_coded_flag) to FALSE ("0") if the block to be coded for the "Y" component in the YCgCo color space does not contain a non-zero transform coefficient.

[0042] The entropy coding unit 107 sets the significant coefficient flag (tu_cb_coded_flag) to TRUE ("1") if the block to be coded of the "Cg" component in the YCgCo color space contains a non-zero transform coefficient, and sets the significant coefficient flag (tu_cb_coded_flag) to FALSE ("0") if the block to be coded of the "Cg" component in the YCgCo color space does not contain a non-zero transform coefficient.

[0043] The entropy coding unit 107 sets the significant coefficient flag (tu_cr_coded_flag) to TRUE ("1") if the block to be coded of the "Co" component in the YCgCo color space contains a non-zero transform coefficient, and sets the significant coefficient flag (tu_cr_coded_flag) to FALSE ("0") if the block to be coded of the "Co" component in the YCgCo color space does not contain a non-zero transform coefficient.

[0044] Furthermore, the entropy coding unit 107 includes a color space conversion application flag (cu_act_enabled_flag) indicating whether or not color space conversion is applied in the bit stream for each block to be coded, and signals the flag to the decoding side. Such a color space conversion flag is also called a color space conversion application flag. The color space conversion application flag is an example of a second flag.

[0045] When the color space conversion application flag is TRUE ("1"), it indicates that color space conversion is applied to the corresponding block to be coded. When the color space conversion application flag is FALSE ("0"), it indicates that color space conversion is not applied to the corresponding block to be coded. Note that the entropy coding unit 107 may use a color space conversion non-application flag instead of the color space conversion application flag. In that case, when the color space conversion non-application flag is TRUE ("1"), it indicates that color space conversion is not applied to the corresponding block to be coded. When the color space conversion non-application flag is FALSE ("0"), it indicates that color space conversion is applied to the corresponding block to be coded.

[0046] The inverse quantization and inverse transform unit 130 performs inverse quantization processing and inverse transform processing on a block-by-block basis for each color component. The inverse quantization and inverse transform unit 130 includes an inverse quantization unit 108 and an inverse transform unit 109.

[0047] The inverse quantization unit 108 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 106. Specifically, the inverse quantization unit 108 restores the quantized transform coefficients output by the quantization unit 106 by inverse quantizing the quantized transform coefficients using a quantization parameter (Qp) and a scaling list, and outputs the restored transform coefficients to the inverse transform unit 109.

[0048] The inverse transform unit 109 performs an inverse transform process corresponding to the transform process performed by the transform unit 105. For example, when the transform unit 105 performs a discrete cosine transform, the inverse transform unit 109 performs an inverse discrete cosine transform. The inverse transform unit 109 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 108 to restore a prediction residual, and outputs a restored prediction residual which is the restored prediction residual.

[0049] The color space inverse conversion unit 110 performs color space inverse conversion, which is the inverse process of the color space conversion performed by the color space conversion unit 104, and outputs the prediction residual after the color space inverse conversion. Specifically, the color space inverse conversion unit 110 performs the following inverse conversion calculation using the Y component, Cg component, and Co component of the restored prediction residual, thereby performing inverse conversion from the YCgCo color space to the RGB color space.

[0050] t=Y-(Cg>>1) G=Cg+t B=t-(Co>>1) R=Co+B

[0051] The scaling unit 111 performs an inverse process (inverse scaling process) corresponding to the chrominance residual scaling performed by the scaling unit 103. Specifically, the scaling unit 111 performs an inverse process (inverse scaling process) corresponding to the chrominance component prediction residual C ResScale Based on this, the prediction residual C before scaling by the scaling unit 103 is Res , C ResScale / C Scale , i.e., C ResScale ×C ScaleInv Calculate and output as follows.

[0052] In this embodiment, the scaling unit 111 controls whether or not chrominance residual scaling is on, based on the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the chrominance components and the color space conversion application flag (cu_act_enabled_flag). The details of the on / off control of the chrominance residual scaling will be described later.

[0053] The synthesis unit 112 synthesizes the restored prediction residual with the prediction block output by the prediction unit 140 on a pixel-by-pixel basis. The synthesis unit 112 adds each pixel value of the restored prediction residual to each pixel value of the prediction block to restore (reconstruct) the block to be coded, and outputs a restored block. The restored block may also be called a reconstructed block.

[0054] The luminance inverse mapping unit 113 performs an inverse mapping process on the restored block, which corresponds to the luminance mapping process performed by the luminance mapping unit 101 .

[0055] The in-loop filter 114 performs filtering on the reconstructed block, and outputs the filtered reconstructed block to the memory 115. The in-loop filter 114 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0056] The memory 115 stores the reconstructed blocks after filtering output by the in-loop filter 114, and accumulates the reconstructed blocks as a reconstructed image on a frame-by-frame basis.

[0057] The prediction unit 140 performs prediction processing on a block-by-block basis for each color component. The prediction unit 140 generates a prediction block for each color component by performing prediction processing such as intra prediction and inter prediction on the block to be coded. The prediction unit 140 includes an intra prediction unit 116, an inter prediction unit 117, a luminance mapping unit 118, and a switching unit 119.

[0058] The intra prediction unit 116 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 116 generates an intra prediction block by referring to restored pixels around the encoding target block, and outputs the generated intra prediction block. The intra prediction unit 116 selects an intra prediction mode to be applied to the encoding target block from among a plurality of intra prediction modes, and predicts the encoding target block using the selected intra prediction mode.

[0059] The inter prediction unit 117 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 117 uses the restored image stored in the memory 115 as a reference image, calculates a motion vector by a method such as block matching, predicts a block to be coded, generates an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 117 selects an optimal inter prediction method from inter prediction using multiple reference images (typically, bi-prediction) and inter prediction using one reference image (unidirectional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 117 outputs information related to the inter prediction (motion vector, etc.) to the entropy coding unit 107.

[0060] The luminance mapping unit 118 performs a luminance mapping process similar to that performed by the luminance mapping unit 101 on the inter prediction block output by the inter prediction unit 117 .

[0061] The switching unit 119 switches between the inter prediction block and the intra prediction block, and outputs either the prediction block to the residual generation unit 102 and the synthesis unit 112.

[0062] <Configuration of the Decryption Device> Next, the decoding device according to this embodiment will be described, focusing on the differences from the encoding device 1. Fig. 3 is a diagram showing the configuration of a decoding device 2 according to this embodiment.

[0063] As shown in FIG. 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transform unit 220, a color space inverse transform unit 203, a synthesis unit 204, a luminance inverse mapping unit 205, an in-loop filter 206, a memory 207, a prediction unit 230, a scaling unit 212, a synthesis unit 213, an in-loop filter 214, a memory 215, and a prediction unit 240.

[0064] The entropy decoding unit 200 decodes a bit stream (input coded data), obtains a quantized transform coefficient corresponding to a block to be decoded, and outputs the obtained quantized transform coefficient to the inverse quantization and inverse transform unit 220. The entropy decoding unit 200 also obtains information on the transform process and the quantization process and information on the prediction process, outputs the information on the transform process and the quantization process to the inverse quantization and inverse transform unit 220, and outputs the information on the prediction process to the prediction unit 230 and the prediction unit 240.

[0065] The entropy decoding unit 200 also acquires a significant coefficient flag indicating whether the block to be decoded includes a non-zero transform coefficient for each of the first color component (the “G” component in the RGB color space, the “Y” component in the YCgCo color space), the second color component (the “B” component in the RGB color space, the “Cg” component in the YCgCo color space), and the third color component (the “R” component in the RGB color space, the “Co” component in the YCgCo color space). Furthermore, the entropy decoding unit 200 acquires a color space conversion application flag (cu_act_enabled_flag) indicating whether or not color space conversion is applied for each block to be decoded.

[0066] The inverse quantization and inverse transform unit 220 performs inverse quantization processing and inverse transform processing on a block-by-block basis for each color component.

[0067] The inverse quantization unit 201 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 106 of the encoding device 1. The inverse quantization unit 201 restores the transform coefficients of the block to be decoded by inverse quantizing the quantized transform coefficients output by the entropy decoding unit 200, using a quantization parameter (Qp) and a scaling list, and outputs the restored transform coefficients to the inverse transform unit 202.

[0068] The inverse transform unit 202 performs inverse transform processing corresponding to the transform processing performed by the transform unit 105 of the encoding device 1. The inverse transform unit 202 performs inverse transform processing on the transform coefficients output by the inverse quantization unit 201 to reconstruct and output prediction residuals.

[0069] When the color space conversion application flag (cu_act_enabled_flag) indicates that the block to be decoded has been coded using color space conversion, the color space inverse conversion unit 203 performs color space inverse conversion processing, which is the inverse processing of the color space conversion performed by the color space conversion unit 104 of the coding device 1, on the restored prediction residual. Specifically, the color space inverse conversion unit 203 performs the following inverse conversion calculations using the Y component, Cg component, and Co component of the restored prediction residual.

[0070] t=Y-(Cg>>1) G=Cg+t B=t-(Co>>1) R=Co+B

[0071] The synthesis unit 204 decodes (reconstructs) the original block by synthesizing the prediction residual of the first color component (the “G” component in the RGB color space, the “Y” component in the YCgCo color space) with the prediction block of the first color component output by the prediction unit 230 on a pixel-by-pixel basis, and outputs a restored block of the first color component. Hereinafter, the first color component is referred to as the luminance component.

[0072] The luminance inverse mapping unit 205 performs an inverse mapping process on the restored block of the luminance component, which corresponds to the luminance mapping process performed by the luminance mapping unit 101 of the encoding device 1 .

[0073] The in-loop filter 206 performs a filter process on the reconstructed block of the luminance component, and outputs the reconstructed block after the filter process to the memory 207. The in-loop filter 206 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0074] The memory 207 stores the reconstructed blocks after filtering output by the in-loop filter 206, and accumulates the reconstructed blocks as a reconstructed image on a frame-by-frame basis.

[0075] The prediction unit 230 performs prediction processing on the luminance component on a block-by-block basis. The prediction unit 230 includes an intra prediction unit 208, an inter prediction unit 209, a luminance mapping unit 210, and a switching unit 211.

[0076] The intra prediction unit 208 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 208 generates an intra prediction block by referring to restored pixels around the block to be decoded, and outputs the generated intra prediction block. The intra prediction unit 208 predicts the block to be decoded using an intra prediction mode signaled from the encoding device 1 from among a plurality of intra prediction modes.

[0077] The inter prediction unit 209 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 209 uses the restored image stored in the memory 207 as a reference image, calculates a motion vector by a method such as block matching, predicts a block to be decoded, generates an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 209 performs inter prediction using an inter prediction method signaled from the encoding device 1 from among inter prediction using multiple reference images and inter prediction using one reference image.

[0078] The luminance mapping unit 210 performs a luminance mapping process similar to that performed by the luminance mapping unit 101 of the encoding device 1 on the inter-prediction block output by the inter prediction unit 209 .

[0079] The switching unit 211 switches between the inter-predicted block and the intra-predicted block, and outputs either the predicted block to the synthesis unit 204.

[0080] On the other hand, the scaling unit 212 performs chrominance residual scaling in which the prediction residual of the chrominance components (second and third color components) is scaled based on the luminance component (first color component) corresponding to the chrominance components, similar to the scaling unit 111 of the encoding device 1. Specifically, the scaling unit 212 scales the prediction residual C ResScale Based on this, the prediction residual C Res , C ResScale / C Scale , i.e., C ResScale ×C ScaleInv Calculate and output as follows.

[0081] In this embodiment, the scaling unit 212 controls whether chrominance residual scaling is on or off, based on the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the chrominance components and the color space conversion application flag (cu_act_enabled_flag) acquired by the entropy decoding unit 200. The details of such on / off control of chrominance residual scaling will be described later.

[0082] The synthesis unit 213 synthesizes the prediction residual of the chrominance component with the prediction block of the chrominance component output by the prediction unit 240 on a pixel-by-pixel basis to decode (reconstruct) the original block, and outputs a restored block of the chrominance component.

[0083] The in-loop filter 214 performs filtering on the reconstructed block of the chrominance component, and outputs the filtered reconstructed block to the memory 215. The in-loop filter 214 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0084] The memory 215 stores the reconstructed blocks after filtering output by the in-loop filter 214, and accumulates the reconstructed blocks as a reconstructed image on a frame-by-frame basis.

[0085] The prediction unit 240 performs prediction processing on the chrominance component on a block-by-block basis. The prediction unit 240 includes an intra prediction unit 216, an inter prediction unit 217, and a switching unit 218.

[0086] The intra prediction unit 216 performs intra prediction using spatial correlation within a frame. Specifically, the intra prediction unit 216 generates an intra prediction block by referring to restored pixels around the block to be decoded, and outputs the generated intra prediction block. The intra prediction unit 216 predicts the block to be decoded using an intra prediction mode signaled from the encoding device 1 from among a plurality of intra prediction modes.

[0087] The inter prediction unit 217 performs inter prediction using correlation between frames. Specifically, the inter prediction unit 217 uses the restored image stored in the memory 215 as a reference image, calculates a motion vector by a method such as block matching, predicts a block to be decoded, generates an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 217 performs inter prediction using an inter prediction method signaled from the encoding device 1 from among inter prediction using multiple reference images and inter prediction using one reference image.

[0088] The switching unit 218 switches between the inter-predicted block and the intra-predicted block, and outputs either the predicted block to the synthesis unit 213.

[0089] <Chrominance residual scaling on / off control> Next, on / off control of chrominance residual scaling according to this embodiment will be described.

[0090] As described above, when color space conversion is applied, the transform coefficients transmitted to the decoding device 2 are in YCgCo color space, and tu_cb_coded_flag is set to TRUE ("1") if there are non-zero transform coefficients for the Cg component, and tu_cr_coded_flag is set to TRUE ("1") if there are non-zero transform coefficients for the Co component.

[0091] If a non-zero transform coefficient exists in any of the color components Y, Cg, or Co, in the decoding device 2, the energy of that transform coefficient is dispersed to each color component in the RGB color space by the color space inverse transform process of the color space inverse transform unit 203, so that there is a high possibility that prediction residuals will occur in all color components in the RGB color space.

[0092] In this embodiment, the scaling unit 212 of the decoding device 2 performs on / off control of chrominance residual scaling in consideration of not only tu_cb_coded_flag and tu_cr_coded_flag but also the color space conversion application flag (cu_act_enabled_flag). Specifically, the scaling unit 212 performs chrominance residual scaling when the color space conversion application flag (cu_act_enabled_flag) is TRUE ("1"), regardless of the significant coefficient flags of the chrominance components (tu_cb_coded_flag, tu_cr_coded_flag). This allows chrominance residual scaling to be correctly applied, improving coding efficiency.

[0093] In other words, the decoding device 2 of this embodiment is a device that performs decoding processing on a block to be decoded obtained by dividing an original image composed of multiple color components, and has an entropy decoding unit 200, an inverse quantization / inverse transform unit 220, a color space inverse transform unit 203, and a scaling unit 212.

[0094] First, the entropy decoding unit 200 decodes the bit stream to output transform coefficients for each color component of the block to be decoded, significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) indicating for each color component whether the block to be decoded includes a non-zero transform coefficient, and a color space conversion application flag (cu_act_enabled_flag) indicating whether the block to be decoded is coded using color space conversion that converts the color space of the prediction residual from the color space of the original image to another color space. Hereinafter, the second color component and the third color component are not particularly distinguished from each other, and each of tu_cb_coded_flag and tu_cr_coded_flag is simply referred to as a "tuCbfChroma flag".

[0095] Second, the inverse quantization and inverse transform unit 220 reconstructs the prediction residual from the transform coefficients for each color component.

[0096] Third, when the color space conversion application flag (cu_act_enabled_flag) indicates that the current block has been coded using color space conversion, the color space inverse conversion unit 203 performs color space inverse conversion on the reconstructed prediction residual.

[0097] Fourth, the scaling unit 212 performs chrominance residual scaling, which scales the prediction residual of the chrominance component based on the luminance component corresponding to the chrominance component. Here, the scaling unit 212 determines whether to perform chrominance residual scaling based on the tuCbfChroma flag and the color space conversion application flag (cu_act_enabled_flag).

[0098] FIG. 4 is a diagram showing on / off control of chrominance residual scaling according to this embodiment.

[0099] 4, when a color space conversion application flag (cu_act_enabled_flag) indicates that the current block has been coded using color space conversion, that is, when cu_act_enabled_flag=TRUE (“1”) (step S1: YES), the scaling unit 212 determines to perform chrominance residual scaling (step S2). In this case, the scaling unit 212 and the synthesis unit 213 perform chrominance block reconstruction processing involving chrominance residual scaling (step S3).

[0100] Furthermore, if the tuCbfChroma flag indicates that the chrominance component decoding target block includes a non-zero transform coefficient, that is, if the tuCbfChroma flag is TRUE (“1”) (step S1: YES), the scaling unit 212 determines to perform chrominance residual scaling (step S2). In this case, the scaling unit 212 and the synthesis unit 213 perform chrominance block reconstruction processing involving chrominance residual scaling (step S3).

[0101] On the other hand, if the tuCbfChroma flag indicates that the current block of the chrominance component does not include a non-zero transform coefficient and the color space conversion application flag (cu_act_enabled_flag) indicates that the current block has not been coded using color space conversion, that is, if the tuCbfChroma flag=FALSE("0") and the cu_act_enabled_flag=FALSE("0") (step S1: NO), the scaling unit 212 determines not to perform chrominance residual scaling (step S4). In this case, the scaling unit 212 and the synthesis unit 213 perform chrominance block reconstruction processing without chrominance residual scaling (step S5).

[0102] In this way, the scaling unit 212 according to this embodiment performs chrominance residual scaling when the color space conversion application flag (cu_act_enabled_flag) is TRUE ("1"), regardless of the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the chrominance components. This allows chrominance residual scaling to be correctly applied, improving the coding efficiency.

[0103] <Other embodiments> A program may be provided that causes a computer to execute each process performed by the encoding device 1. Also, a program may be provided that causes a computer to execute each process performed by the decoding device 2. The program may be recorded in a computer-readable medium. Using the computer-readable medium, it is possible to install the program in a computer. Here, the computer-readable medium on which the program is recorded may be a non-transient recording medium. The non-transient recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0104] The circuits for executing the processes performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured as a semiconductor integrated circuit (chip set, SoC). The circuits for executing the processes performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured as a semiconductor integrated circuit (chip set, SoC).

[0105] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes, etc. are possible without departing from the spirit of the invention. [Explanation of symbols]

[0106] 1: Encoding device 2: Decryption device 100: Block division section 101: Luminance mapping section 102:Residual generator 103: Scaling section 104: Color space conversion unit 105: Conversion section 106:Quantization section 107: Entropy coding unit 108: Inverse quantization section 109: Reverse conversion section 110: Color space inverse conversion unit 111: Scaling section 112: Synthesis section 113: Luminance inverse mapping unit 114: In-loop filter 115: Memory 116: Intra prediction unit 117: Inter prediction section 118: Luminance mapping section 119: Switching section 120: Transformation and quantization section 130: Inverse quantization and inverse transformation unit 140: Prediction Department 200: Entropy decoding unit 201: Inverse quantization section 202: Inverse conversion unit 203: Color space inverse conversion unit 204: Composite section 205: Luminance inverse mapping unit 206: In-loop filter 207: Memory 208: Intra prediction unit 209: Inter prediction unit 210: Luminance mapping section 211: Switching section 212: Scaling section 213: Composite section 214: In-loop filter 215: Memory 216: Intra prediction unit 217: Inter prediction unit 218: Switching section 220: Inverse quantization and inverse transformation unit 230: Prediction Department 240: Prediction Department

Claims

1. A decoding device that performs a decoding process on blocks obtained by dividing an original image composed of a plurality of color components, an entropy decoding unit that decodes a bitstream to output a transform coefficient for each color component of the block, a first flag indicating, for each color component, whether the block includes a non-zero transform coefficient, and a second flag indicating whether a prediction residual transform process that generates a new prediction residual for each color component by a process using prediction residuals of two or more color components has been applied to the block; an inverse quantization and inverse transform unit that reconstructs the prediction residual from the transform coefficients for each color component; a prediction residual transform processing unit that performs a prediction residual transform processing on the prediction residual when the second flag indicates that the prediction residual transform processing has been applied to the block; a scaling unit that performs chrominance residual scaling by scaling the prediction residual of a chrominance component based on a luminance component corresponding to the chrominance component, The scaling unit determines whether to perform the chrominance residual scaling based on the second flag when the first flag of the chrominance component indicates that the chrominance component includes the zero transform coefficient. Decryption device.

2. A program that causes a computer to function as the decoding device according to claim 1.

3. A decoding method for performing a decoding process on blocks obtained by dividing an original image composed of a plurality of color components, outputting, by decoding the bitstream, transform coefficients for each color component of the block, a first flag indicating for each color component whether the block includes a non-zero transform coefficient, and a second flag indicating whether a prediction residual transform process has been applied to the block, the prediction residual transform process generating a new prediction residual for each color component by a process using prediction residuals of two or more color components; performing a prediction residual transform process on the prediction residual reconstructed from the transform coefficients if the second flag indicates that the prediction residual transform process has been applied to the block; and when the first flag for a chrominance component indicates that the chrominance component includes a zero transform coefficient, determining, based on the second flag, whether to perform chrominance residual scaling, in which the prediction residual for the chrominance component is scaled based on a luminance component corresponding to the chrominance component.