Decryption device, program, and decryption method

Abstract

To appropriately apply color difference residual scaling to improve coding efficiency.SOLUTION: A decoding method includes the steps of outputting a conversion coefficient for each color component of a block, a first flag indicating whether the block contains a non-zero conversion coefficient for each color component, and a second flag indicating whether the block is encoded by using a color space conversion that converts a color space of the predicted residual from a color space of the original image to another color space by decoding a bit stream, performing color space inverse conversion on the predicted residual restored from the conversion coefficient when the second flag indicates that the block is encoded using the color space conversion, and determining whether to perform color difference residual scaling in which the predicted residual of the color difference component is scaled on the basis of a luminance component corresponding to the color difference component on the basis of the first flag of the color difference component and the second flag.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a decoding apparatus, a program, and a decoding method.

Background Art

[0002] Non-Patent Document 1 defines an Adaptive Colour Transform (ACT) for encoding RGB4:4:4 video in Versatile Video Coding (VVC). The colour space conversion is a technique for removing the correlation between colour components of the prediction residual by converting the prediction residual in the RGB colour space to the YCgCo colour space, thereby improving the encoding efficiency.

[0003] The encoding apparatus performs an orthogonal transform for each colour component (Y, Cg, Co components) on the prediction residual converted to the YCgCo colour space, quantizes and entropy-encodes the transform coefficients, and outputs a stream. On the decoding apparatus side, the transmitted transform coefficients are entropy-decoded, and an inverse colour space transform (inverse ACT) is performed on the prediction residual in the YCgCo colour space obtained by inverse quantization and inverse orthogonal transform to convert it to the prediction residual in the RGB colour space, and the decoded image is obtained by synthesizing the prediction residual with the predicted image.

[0004] By the way, in VVC, a technique called Chroma Residual Scaling (CRS) is adopted, in which the prediction residual of the chrominance component is scaled according to the corresponding luminance component.

[0005] The decoding apparatus controls whether to apply chroma residual scaling based on significance coefficient flags (tu_cb_coded_flag and tu_cr_coded_flag) indicating whether non-zero transform coefficients of the chrominance components are transmitted, in order to reduce the computational amount of chroma residual scaling. Specifically, the decoding apparatus performs chroma residual scaling only when the significance coefficient flag indicates that non-zero transform coefficients of the chrominance components are transmitted.

Prior Art Documents

Non-Patent Documents

[0006] [Non-Patent Document 1] JVET-R2001 “Versatile Video Coding Draft 9” [Summary of the Invention] [Problems to be Solved by the Invention]

[0007] When color space conversion is applied, the conversion coefficients transmitted to the decoder are in the YCgCo color space. tu_cb_coded_flag is set to TRUE ("1") when there are non-zero conversion coefficients for the Cg component, and tu_cr_coded_flag is set to TRUE ("1") when there are non-zero conversion coefficients for the Co component.

[0008] When there are non-zero conversion coefficients in any of the color components of Y, Cg, or Co, in the decoder, since their energy is dispersed to each color component in the RGB color space by inverse color space conversion, prediction residuals are likely to occur in all color components in the RGB color space.

[0009] However, in Non-Patent Document 1, the decoder performs on / off control of chrominance residual scaling based on tu_cb_coded_flag and tu_cr_coded_flag. For this reason, when the significant coefficient flag of the chrominance component is FALSE ("0"), the decoder does not perform chrominance residual scaling even when prediction residuals occur in all color components of RGB due to inverse color space conversion. For this reason, there is a problem that chrominance residual scaling is not correctly applied and the coding efficiency is reduced.

[0010] Therefore, an object of the present invention is to provide a decoder, a program, and a decoding method that improve coding efficiency by appropriately applying chrominance residual scaling. [Means for Solving the Problems]

[0011] The decoding apparatus according to the first aspect is a decoding apparatus that performs decoding processing on blocks obtained by dividing an original image composed of a plurality of color components. By decoding a bit stream, the conversion coefficients for each color component of the block, a first flag indicating for each color component whether the block includes non-zero conversion coefficients, and a second flag indicating whether the block is encoded using a color space conversion that converts the color space of the prediction residual from the color space of the original image to another color space are output by an entropy decoding unit. An inverse quantization and inverse transformation unit restores the prediction residual from the conversion coefficients for each color component. When the second flag indicates that the block is encoded using the color space conversion, a color space inverse conversion unit performs a color space inverse conversion on the prediction residual. A scaling unit performs color difference residual scaling that scales the prediction residual of the color difference component based on the luminance component corresponding to the color difference component. The gist is that the scaling unit determines whether to perform the color difference residual scaling based on the first flag of the color difference component and the second flag.

[0012] The program according to the second aspect causes a computer to function as the decoding apparatus according to the first aspect.

[0013] The decoding method according to the third aspect is a decoding method that performs decoding processing on blocks obtained by dividing an original image composed of a plurality of color components. By decoding a bit stream, a step of outputting the conversion coefficients for each color component of the block, a first flag indicating for each color component whether the block includes non-zero conversion coefficients, and a second flag indicating whether the block is encoded using a color space conversion that converts the color space of the prediction residual from the color space of the original image to another color space is performed. When the second flag indicates that the block is encoded using the color space conversion, a step of performing a color space inverse conversion on the prediction residual restored from the conversion coefficients is performed. A step of determining whether to perform color difference residual scaling that scales the prediction residual of the color difference component based on the luminance component corresponding to the color difference component based on the first flag of the color difference component and the second flag is included. The gist is that it has these steps.

Effect of the Invention

[0014] According to the present invention, it is possible to provide a decoding device, a program, and a decoding method that improve the encoding efficiency by appropriately applying color difference residual scaling.

Brief Description of the Drawings

[0015]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0016] With reference to the drawings, an encoding device and a decoding device according to an embodiment will be described. The encoding device and the decoding device according to the embodiment respectively perform encoding and decoding of moving images represented by MPEG (Moving Picture Experts Group). In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals.

[0017] <Configuration of Encoding Device> First, the configuration of the encoding device according to the present embodiment will be described. FIG. 1 is a diagram showing the configuration of the encoding device 1 according to the present embodiment.

[0018] As shown in FIG. 1, the encoding device 1 includes a block division unit 100, a luminance mapping unit 101, a residual generation unit 102, a scaling unit 103, a color space conversion unit 104, a conversion / quantization unit 120, an entropy encoding unit 107, an inverse quantization / inverse conversion unit 130, a color space inverse conversion unit 110, a scaling unit 111, a synthesis unit 112, a luminance inverse mapping unit 113, an in-loop filter 114, a memory 115, and a prediction unit 140.

[0019] The block division unit 100 divides the original image, which is an input image in units of frames (or pictures) constituting the moving image, into a plurality of image blocks, and outputs the image blocks obtained by the division to the residual generation unit 102. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels, etc. The shape of the image block is not limited to a square and may be a rectangle (non-square). The image block is a unit in which the encoding device 1 performs encoding processing (i.e., the encoding target block), and is also a unit in which the decoding device performs decoding processing (i.e., the decoding target block). Such an image block is sometimes called a CU (Coding Unit).

[0020] The input image is an RGB signal, and the chroma format may be 4:4:4. The RGB color space is an example of the first color space. The "G" component corresponds to the first color component, the "B" component corresponds to the second color component, and the "R" component corresponds to the third color component. The block division unit 100 performs block division on each of the R component, G component, and B component constituting the image, and outputs blocks for each color component. In the following description of the encoding device, when each color component is not distinguished, it is simply called the encoding target block.

[0021] The luminance mapping unit 101 performs mapping processing on each pixel value in the encoding target block of the luminance component output by the block division unit 100 based on a mapping table, and generates and outputs an encoding target block of the new luminance component obtained by the mapping.

[0022] FIG. 2 is a graph showing an example of the relationship between the input pixel value and the output pixel value in the luminance mapping process according to the present embodiment. In FIG. 2, the horizontal axis represents the value of the input signal, and the vertical axis represents the value of the output signal.

[0023] As shown in FIG. 2, the mapping table is a table set for one or more slices, and is a coefficient table for representing the relationship between the input signal before the mapping process and the output signal after the mapping process. Specifically, in the mapping table, for each band obtained by dividing in advance the range from the minimum value to the maximum value that the input signal (pixel value to be mapped) can take before mapping into a prescribed number (N), a value indicating the number of pixel values of the output signal after conversion to be assigned is stored.

[0024] For example, the mapping table will be described by taking as an example the case where the number N of bands is 16 in the mapping process of a 10-bit image signal. The range from the minimum value 0 to the maximum value 1023 that the input signal can take before mapping is assigned as the input signal corresponding to each equally divided band. For example, the first band corresponds to input pixel values from 0 to 63. Also, the second band corresponds to input pixel values from 64 to 127. Similarly, the input signal is assigned up to the 16th band.

[0025] Each band corresponds to the position of each coefficient in the mapping table. The coefficient stored in the mapping table means the number of output pixel values assigned to each band. For example, for the mapping table lmcs CW ={39, 40, 55, 70, 80, 90, 97, 97, 104, 83, 57, 55, 49, 44, 34, 30}, the output pixel values corresponding to the first band are from 0 to 38, and the output pixel values corresponding to the second band are from 39 to 78. The third to 16th bands are assigned in the same way. When the value corresponding to a certain band in the mapping table is large, the number of output pixel values assigned to that band increases, and conversely, when it is small, the number of output pixel values assigned to that band decreases.

[0026] The mapping table may be set in the encoding device 1 according to the occurrence frequency of the luminance signal values of one or more slices, or may be selected by the encoding device 1 from a plurality of mapping tables defined in advance in the system, or a mapping table defined in advance in the system may be used. Note that the mapping table may store a value indicating the number of pixel values of the input signal before conversion to be assigned to each band obtained by dividing in advance the range from the minimum value to the maximum value that the output signal after mapping can take into a prescribed number, or the values in the mapping table may be quantized and held. As long as the relationship between the input signal and the output signal before and after mapping is shown, it is not limited to the above example.

[0027] Also, when the encoding device 1 sets a mapping table according to the occurrence frequency of the luminance signal values or selects from a plurality of mapping tables, the encoding device 1 transmits the mapping table information to the decoding device 2 by some means. For example, the encoding device 1 may entropy-encode the information of the table values and output a stream. Further, a mapping table prepared in advance by the encoding device 1 and the decoding device 2 may be switched and used based on video format information (for example, parameters representing the relationship between the optical signal and the electrical signal in the video signal).

[0028] The residual generation unit 102 calculates a prediction residual representing the difference (error) between the encoding target block output by the block division unit 100 and the prediction block obtained by predicting the encoding target block by the prediction unit 140. Specifically, the residual generation unit 102 calculates the prediction residual by subtracting each pixel value of the prediction block from each pixel value of the encoding target block for each color component, and outputs the calculated prediction residual. That is, the residual generation unit 102 generates the prediction residual for each color component based on the difference between the encoding target block for each color component and the prediction block for each color component.

[0029] The scaling unit 103 performs color difference residual scaling on the prediction residual of the color difference component output by the residual generation unit 102. Color difference residual scaling is a process of scaling the prediction residual of the color difference component according to the corresponding luminance component. Note that when the luminance mapping unit 101 does not perform luminance mapping, the color difference residual scaling becomes invalid.

[0030] The color difference residual scaling depends on the average value of the decoded adjacent luminance pixel values above and / or to the left of the block to be encoded. The scaling unit 103 obtains the index Y from the average value avgYr of the decoded adjacent luminance pixel values Idx and obtains the scaling coefficient C ScaleInv using cScaleInv[Y Idx . Here, cScaleInv[] is a look-up table. The scaling unit 103 performs color difference residual scaling on the entire block to be encoded of the color difference component, while luminance mapping is performed for each pixel value. Specifically, when the prediction residual of the color difference component is C Res , the prediction residual C ResScale of the color difference component after scaling is calculated and output as C Res * C Scale , that is, C Res / C ScaleInv .

[0031] The color space conversion unit 104 performs color space conversion on the prediction residual of each color component and outputs the prediction residual after color space conversion. For example, the color space conversion unit 104 generates the prediction residual in the YCgCo color space by performing the following conversion calculations on the R component, G component, and B component of the prediction residual of the block to be encoded.

[0032] Co = R - B t = B + (Co >> 1) Cg = G - t Y = t + (Cg >> 1)

[0033] However, ">>" represents a right shift operation. Also, the "Y" component corresponds to the first color component, the "Cg" component corresponds to the second color component, and the "Co" component corresponds to the third color component. Such a YCgCo color space is an example of the second color space.

[0034] Note that the color space conversion in the color space conversion unit 104 may generate a prediction residual composed of new color components by addition, subtraction, multiplication, division, shift processing, etc. for each color component. Also, the color space conversion does not necessarily have to be a conversion that affects all color components. For example, the color space conversion unit 104 may apply a color space conversion in which the first color component is maintained without change, the average value of the second color component and the third color component is used as the new second color component, and the difference between the second color component and the third color component is used as the new third color component.

[0035] The conversion and quantization unit 120 performs conversion processing and quantization processing for each color component in block units. The conversion and quantization unit 120 includes a conversion unit 105 and a quantization unit 106.

[0036] The conversion unit 105 performs conversion processing on the prediction residual (referred to as the prediction residual regardless of whether color space conversion is applied) to calculate conversion coefficients and outputs the calculated conversion coefficients. Specifically, the conversion unit 105 generates conversion coefficients for each color component by performing conversion processing on the prediction residual of each color component in block units. The conversion processing may be a frequency conversion such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or discrete wavelet transform. Also, the conversion unit 105 outputs information regarding the conversion processing to the entropy encoding unit 107.

[0037] The conversion processing includes a conversion skip in which no conversion processing is performed. The conversion skip also includes a conversion that applies conversion processing only horizontally or a conversion that applies conversion processing only vertically. Also, the conversion unit 105 may perform a secondary conversion processing in which further conversion processing is applied to the conversion coefficients obtained by the conversion processing. The secondary conversion processing may be applied only to a partial region of the conversion coefficients.

[0038] The quantization unit 106 quantizes the conversion coefficients output by the conversion unit 105 using quantization parameters and a scaling list, and outputs the quantized conversion coefficients. Further, the quantization unit 106 outputs information regarding the quantization process (specifically, information on the quantization parameters and the scaling list used in the quantization process) to the entropy encoding unit 107 and the inverse quantization unit 108.

[0039] The entropy encoding unit 107 performs entropy encoding on the quantized conversion coefficients output by the quantization unit 106, performs data compression to generate a bit stream (encoded data), and outputs the bit stream to the decoding side. For entropy encoding, Huffman coding, CABAC (Context-based Adaptive Binary Arithmetic Coding), or the like can be used. Further, the entropy encoding unit 107 includes information regarding the conversion process from the conversion unit 105 in the bit stream and signals it to the decoding side, or includes information regarding the prediction process from the prediction unit 140 in the bit stream and signals it to the decoding side.

[0040] Further, the entropy encoding unit 107 includes in the bit stream a significant coefficient flag indicating whether or not the encoding target block includes non-zero conversion coefficients for each of the first color component (the "G" component in the RGB color space, the "Y" component in the YCgCo color space), the second color component (the "B" component in the RGB color space, the "Cg" component in the YCgCo color space), and the third color component (the "R" component in the RGB color space, the "Co" component in the YCgCo color space), and signals it to the decoding side. The significant coefficient flag is an example of the first flag.

[0041] For example, when the block to be coded of the "Y" component in the YCgCo color space contains non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_y_coded_flag) to TRUE ("1"), and when the block to be coded of the "Y" component in the YCgCo color space does not contain non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_y_coded_flag) to FALSE ("0").

[0042] When the block to be coded of the "Cg" component in the YCgCo color space contains non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_cb_coded_flag) to TRUE ("1"), and when the block to be coded of the "Cg" component in the YCgCo color space does not contain non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_cb_coded_flag) to FALSE ("0").

[0043] When the block to be coded of the "Co" component in the YCgCo color space contains non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_cr_coded_flag) to TRUE ("1"), and when the block to be coded of the "Co" component in the YCgCo color space does not contain non-zero transform coefficients, the entropy coding unit 107 sets the significant coefficient flag (tu_cr_coded_flag) to FALSE ("0").

[0044] Furthermore, the entropy coding unit 107 includes the color space transform application flag (cu_act_enabled_flag) indicating whether or not the color space transform is applied in the bit stream for each block to be coded and signals it to the decoding side. Such a color space transform flag is also called a color space transform application flag. The color space transform application flag is an example of the second flag.

[0045] When the color space conversion application flag is TRUE ("1"), it indicates that color space conversion is applied to the corresponding block to be encoded. When the color space conversion application flag is FALSE ("0"), it indicates that color space conversion is not applied to the corresponding block to be encoded. Note that the entropy encoding unit 107 may use a color space conversion non-application flag instead of the color space conversion application flag. In that case, when the color space conversion non-application flag is TRUE ("1"), it indicates that color space conversion is not applied to the corresponding block to be encoded. When the color space conversion non-application flag is FALSE ("0"), it indicates that color space conversion is applied to the corresponding block to be encoded.

[0046] The inverse quantization and inverse transformation unit 130 performs inverse quantization processing and inverse transformation processing for each color component in block units. The inverse quantization and inverse transformation unit 130 includes an inverse quantization unit 108 and an inverse transformation unit 109.

[0047] The inverse quantization unit 108 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 106. Specifically, the inverse quantization unit 108 restores the transform coefficients by inverse quantizing the quantized transform coefficients output by the quantization unit 106 using the quantization parameter (Qp) and the scaling list, and outputs the restored transform coefficients to the inverse transformation unit 109.

[0048] The inverse transformation unit 109 performs inverse transformation processing corresponding to the transformation processing performed by the transformation unit 105. For example, when the transformation unit 105 performs a discrete cosine transform, the inverse transformation unit 109 performs an inverse discrete cosine transform. The inverse transformation unit 109 performs inverse transformation processing on the transform coefficients output by the inverse quantization unit 108 to restore the prediction residual, and outputs the restored prediction residual, which is the restored prediction residual.

[0049] The color space inverse transformation unit 110 performs a color space inverse transformation, which is the inverse process of the color space transformation performed by the color space transformation unit 104, and outputs the prediction residual after the color space inverse transformation. Specifically, the color space inverse transformation unit 110 performs the following inverse transformation calculation using the Y component, Cg component, and Co component of the restored prediction residual to perform an inverse transformation from the YCgCo color space to the RGB color space.

[0050] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R = Co + B

[0051] Scaling unit 111 performs an inverse process (inverse scaling process) corresponding to the color difference residual scaling performed by scaling unit 103. Specifically, scaling unit 111 calculates and outputs the prediction residual C before scaling by scaling unit 103 based on the prediction residual C of the color difference component after scaling by scaling unit 103, that is, C / C, or C × C. ResScale based on Res to ResScale / Scale , that is, ResScale × ScaleInv

[0052] In this embodiment, scaling unit 111 performs on / off control of color difference residual scaling based on the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the color difference components and the color space conversion application flag (cu_act_enabled_flag). Details of such on / off control of color difference residual scaling will be described later.

[0053] Combining unit 112 combines the restored prediction residual with the prediction block output by prediction unit 140 on a pixel-by-pixel basis. Combining unit 112 adds each pixel value of the restored prediction residual and each pixel value of the prediction block to restore (reconstruct) the block to be coded and outputs the restored block. Note that the restored block may also be called the reconstructed block.

[0054] Luminance inverse mapping unit 113 performs an inverse mapping process corresponding to the luminance mapping process performed by luminance mapping unit 101 on the restored block.

[0055] ​The in-loop filter 114 performs a filtering process on the restored block and outputs the restored block after the filtering process to the memory 115. The in-loop filter 114 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0056] The memory 115 stores the restored block after the filtering process output by the in-loop filter 114 and accumulates the restored blocks as a restored image in units of frames.

[0057] The prediction unit 140 performs a prediction process in units of blocks for each color component. The prediction unit 140 generates a prediction block for each color component by performing prediction processes such as intra prediction and inter prediction on the block to be encoded. The prediction unit 140 includes an intra prediction unit 116, an inter prediction unit 117, a luminance mapping unit 118, and a switching unit 119.

[0058] The intra prediction unit 116 performs intra prediction using the spatial correlation within the frame. Specifically, the intra prediction unit 116 generates an intra prediction block by referring to the restored pixels around the block to be encoded and outputs the generated intra prediction block. The intra prediction unit 116 selects an intra prediction mode to be applied to the block to be encoded from among a plurality of intra prediction modes and predicts the block to be encoded using the selected intra prediction mode.

[0059] The inter prediction unit 117 performs inter prediction using the correlation between frames. Specifically, the inter prediction unit 117 uses the restored image stored in the memory 115 as a reference image, calculates a motion vector by a method such as block matching, predicts the block to be encoded to generate an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 117 selects an optimal inter prediction method from among inter predictions using a plurality of reference images (typically, bi-prediction) and inter predictions using one reference image (uni-directional prediction), and performs inter prediction using the selected inter prediction method. The inter prediction unit 117 outputs information related to inter prediction (such as a motion vector) to the entropy encoding unit 107.

[0060] The luminance mapping unit 118 performs luminance mapping processing similar to that of the luminance mapping unit 101 on the inter prediction block output by the inter prediction unit 117.

[0061] The switching unit 119 switches between the inter prediction block and the intra prediction block, and outputs one of the prediction blocks to the residual generation unit 102 and the synthesis unit 112.

[0062] <Configuration of the decoding device> Next, the differences between the decoding device according to the present embodiment and the encoding device 1 will be mainly described. FIG. 3 is a diagram showing the configuration of the decoding device 2 according to the present embodiment.

[0063] As shown in FIG. 3, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization / inverse transformation unit 220, a color space inverse transformation unit 203, a synthesis unit 204, a luminance inverse mapping unit 205, an in-loop filter 206, a memory 207, a prediction unit 230, a scaling unit 212, a synthesis unit 213, an in-loop filter 214, a memory 215, and a prediction unit 240.

[0064] The entropy decoding unit 200 decodes a bit stream (input encoded data), obtains quantization conversion coefficients corresponding to a block to be decoded, and outputs the obtained quantization conversion coefficients to the inverse quantization / inverse transformation unit 220. Further, the entropy decoding unit 200 obtains information related to the conversion process and the quantization process and information related to the prediction process, outputs the information related to the conversion process and the quantization process to the inverse quantization / inverse transformation unit 220, and outputs the information related to the prediction process to the prediction unit 230 and the prediction unit 240.

[0065] Also, the entropy decoding unit 200 obtains a significant coefficient flag indicating whether or not the block to be decoded includes non-zero conversion coefficients for each of the first color component (the "G" component in the RGB color space, the "Y" component in the YCgCo color space), the second color component (the "B" component in the RGB color space, the "Cg" component in the YCgCo color space), and the third color component (the "R" component in the RGB color space, the "Co" component in the YCgCo color space). Further, the entropy decoding unit 200 obtains a color space conversion application flag (cu_act_enabled_flag) indicating whether or not color space conversion is applied for each block to be decoded.

[0066] The inverse quantization / inverse transformation unit 220 performs inverse quantization processing and inverse transformation processing in block units for each color component. The inverse quantization / inverse transformation unit 220 includes an inverse quantization unit 201 and an inverse transformation unit 202.

[0067] The inverse quantization unit 201 performs an inverse quantization process corresponding to the quantization process performed by the quantization unit 106 of the encoding apparatus 1. The inverse quantization unit 201 restores the conversion coefficients of the block to be decoded by inverse quantizing the quantization conversion coefficients output by the entropy decoding unit 200 using a quantization parameter (Qp) and a scaling list, and outputs the restored conversion coefficients to the inverse transformation unit 202.

[0068] The inverse transformation unit 202 performs an inverse transformation process corresponding to the transformation process performed by the transformation unit 105 of the encoding apparatus 1. The inverse transformation unit 202 performs an inverse transformation process on the conversion coefficients output by the inverse quantization unit 201 to restore and output a prediction residual.

[0069] When the color space inverse conversion unit 203 is indicated by the color space conversion application flag (cu_act_enabled_flag) that the block to be decoded is encoded using color space conversion, the color space inverse conversion unit 203 performs a color space inverse conversion process, which is the inverse process of the color space conversion performed by the color space conversion unit 104 of the encoding apparatus 1, on the restored prediction residual. Specifically, the color space inverse conversion unit 203 performs the following inverse conversion calculation using the Y component, Cg component, and Co component of the restored prediction residual.

[0070] t = Y - (Cg >> 1) G = Cg + t B = t - (Co >> 1) R = Co + B

[0071] The synthesizing unit 204 decodes (reconstructs) the original block by synthesizing the prediction residual of the first color component (the "G" component in the RGB color space, the "Y" component in the YCgCo color space) with the prediction block of the first color component output by the prediction unit 230 on a pixel-by-pixel basis, and outputs the restored block of the first color component. Hereinafter, the first color component is referred to as the luminance component.

[0072] The luminance inverse mapping unit 205 performs an inverse mapping process corresponding to the luminance mapping process performed by the luminance mapping unit 101 of the encoding apparatus 1 on the restored block of the luminance component.

[0073] The in-loop filter 206 performs a filtering process on the restored block of the luminance component and outputs the restored block after the filtering process to the memory 207. The in-loop filter 206 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0074] The memory 207 stores the restored block after the filtering process output by the in-loop filter 206 and accumulates the restored blocks as a restored image in units of frames.

[0075] The prediction unit 230 performs prediction processing on the luminance component in units of blocks. The prediction unit 230 includes an intra prediction unit 208, an inter prediction unit 209, a luminance mapping unit 210, and a switching unit 211.

[0076] The intra prediction unit 208 performs intra prediction using the spatial correlation within a frame. Specifically, the intra prediction unit 208 generates an intra prediction block by referring to the restored pixels around the block to be decoded, and outputs the generated intra prediction block. The intra prediction unit 208 predicts the block to be decoded using the intra prediction mode signaled from the encoding device 1 from among a plurality of intra prediction modes.

[0077] The inter prediction unit 209 performs inter prediction using the correlation between frames. Specifically, the inter prediction unit 209 uses the restored image stored in the memory 207 as a reference image, calculates a motion vector by a method such as block matching, predicts the block to be decoded to generate an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 209 performs inter prediction using the inter prediction method signaled from the encoding device 1 from among inter predictions using a plurality of reference images and inter predictions using one reference image.

[0078] The luminance mapping unit 210 performs luminance mapping processing similar to that of the luminance mapping unit 101 of the encoding device 1 on the inter prediction block output by the inter prediction unit 209.

[0079] The switching unit 211 switches between the inter prediction block and the intra prediction block, and outputs one of the prediction blocks to the synthesis unit 204.

[0080] On the other hand, similarly to the scaling unit 111 of the encoding device 1, the scaling unit 212 performs color difference residual scaling for scaling the prediction residuals of the color difference components (second color component, third color component) based on the luminance component (first color component) corresponding to the color difference components. Specifically, the scaling unit 212 calculates and outputs the prediction residual C of the color difference component based on the prediction residual C of the color difference component by C ResScale Res / C ResScale Scale , that is, C ResScale × C ScaleInv . ResScale Based on Res the prediction residual C of the color difference component, the prediction residual C of the color difference component ResScale is calculated and output as C ResScale / C Scale , that is, C ResScale × C ScaleInv . Scale That is, ResScale C ScaleInv × C

[0081] In this embodiment, the scaling unit 212 performs on / off control of the color difference residual scaling based on the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the color difference components and the color space conversion application flag (cu_act_enabled_flag) acquired by the entropy decoding unit 200. Details of such on / off control of the color difference residual scaling will be described later.

[0082] The synthesis unit 213 decodes (reconstructs) the original block by synthesizing the prediction residual of the color difference component with the prediction block of the color difference component output by the prediction unit 240 on a pixel-by-pixel basis, and outputs the restored block of the color difference component.

[0083] The in-loop filter 214 performs a filtering process on the restored block of the color difference component and outputs the restored block after the filtering process to the memory 215. The in-loop filter 214 includes a deblocking filter, a sample adaptive offset (SAO), and the like.

[0084] The memory 215 stores the restored block after the filtering process output by the in-loop filter 214 and accumulates the restored block as a restored image in units of frames.

[0085] The prediction unit 240 performs a prediction process on the color difference components in units of blocks. The prediction unit 240 includes an intra prediction unit 216, an inter prediction unit 217, and a switching unit 218.

[0086] The intra prediction unit 216 performs intra prediction using the spatial correlation within a frame. Specifically, the intra prediction unit 216 generates an intra prediction block by referring to the restored pixels around the block to be decoded, and outputs the generated intra prediction block. The intra prediction unit 216 predicts the block to be decoded using the intra prediction mode signaled from the encoding device 1 among a plurality of intra prediction modes.

[0087] The inter prediction unit 217 performs inter prediction using the correlation between frames. Specifically, the inter prediction unit 217 calculates a motion vector by a method such as block matching using the restored image stored in the memory 215 as a reference image, predicts the block to be decoded to generate an inter prediction block, and outputs the generated inter prediction block. Here, the inter prediction unit 217 performs inter prediction using the inter prediction method signaled from the encoding device 1 among the inter prediction using a plurality of reference images and the inter prediction using one reference image.

[0088] The switching unit 218 switches between the inter prediction block and the intra prediction block, and outputs one of the prediction blocks to the synthesis unit 213.

[0089] <On / Off Control of Chrominance Residual Scaling> Next, the on / off control of chrominance residual scaling according to the present embodiment will be described.

[0090] As described above, when color space conversion is applied, the conversion coefficient transmitted to the decoding device 2 is in the YCgCo color space, tu_cb_coded_flag is set to TRUE (“1”) when there is a non-zero conversion coefficient for the Cg component, and tu_cr_coded_flag is set to TRUE (“1”) when there is a non-zero conversion coefficient for the Co component.

[0091] When there are non-zero conversion coefficients in any of the color components of Y, Cg, and Co, in the decoder device 2, since their energies are dispersed to each color component in the RGB color space by the color space inverse conversion process of the color space inverse conversion unit 203, there is a high possibility that prediction residuals will occur in all color components of the RGB color space.

[0092] In this embodiment, the scaling unit 212 of the decoder device 2 performs on / off control of chrominance residual scaling in consideration of not only the tu_cb_coded_flag and tu_cr_coded_flag but also the color space conversion application flag (cu_act_enabled_flag). Specifically, regardless of the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the chrominance components, when the color space conversion application flag (cu_act_enabled_flag) is TRUE ("1"), the scaling unit 212 performs chrominance residual scaling. Thereby, chrominance residual scaling is correctly applied, and the coding efficiency can be improved.

[0093] That is, the decoder device 2 according to this embodiment is a device that performs decoding processing on a decoding target block obtained by dividing an original image composed of a plurality of color components, and includes an entropy decoding unit 200, an inverse quantization / inverse transformation unit 220, a color space inverse conversion unit 203, and a scaling unit 212.

[0094] First, the entropy decoding unit 200 decodes the bit stream to output the conversion coefficient for each color component of the decoding target block, the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) indicating whether the decoding target block includes non-zero conversion coefficients for each color component, and the color space conversion application flag (cu_act_enabled_flag) indicating whether the decoding target block is encoded using a color space conversion that converts the color space of the prediction residual from the color space of the original image to another color space. Hereinafter, without particularly distinguishing between the second color component and the third color component, each of the tu_cb_coded_flag and tu_cr_coded_flag is simply referred to as the "tuCbfChroma flag".

[0095] Second, the inverse quantization and inverse transformation unit 220 restores the prediction residual from the conversion coefficients for each color component.

[0096] Third, when the color space inverse transformation flag (cu_act_enabled_flag) indicates that the block to be decoded is encoded using color space transformation, the color space inverse transformation unit 203 performs color space inverse transformation on the restored prediction residual.

[0097] Fourth, the scaling unit 212 performs color difference residual scaling for scaling the prediction residual of the color difference component based on the luminance component corresponding to the color difference component. Here, the scaling unit 212 determines whether to perform color difference residual scaling based on the tuCbfChroma flag and the color space transformation application flag (cu_act_enabled_flag).

[0098] FIG. 4 is a diagram showing on / off control of color difference residual scaling according to the present embodiment.

[0099] As shown in FIG. 4, when the color space transformation application flag (cu_act_enabled_flag) indicates that the block to be decoded is encoded using color space transformation, that is, when cu_act_enabled_flag = TRUE ("1") (step S1: YES), the scaling unit 212 determines to perform color difference residual scaling (step S2). In this case, the scaling unit 212 and the synthesis unit 213 perform color difference block reconstruction processing involving color difference residual scaling (step S3).

[0100] Also, when the tuCbfChroma flag indicates that the block to be decoded of the color difference component includes a non-zero conversion coefficient, that is, when the tuCbfChroma flag = TRUE ("1") (step S1: YES), the scaling unit 212 determines to perform color difference residual scaling (step S2). In this case, the scaling unit 212 and the synthesis unit 213 perform color difference block reconstruction processing involving color difference residual scaling (step S3).

[0101] In contrast, when the tuCbfChroma flag indicates that the decoding target block of the chrominance component does not contain non-zero transform coefficients, and the color space transform application flag (cu_act_enabled_flag) indicates that the decoding target block is not encoded using color space transform, that is, when the tuCbfChroma flag = FALSE ("0") and the cu_act_enabled_flag = FALSE ("0") (step S1: NO), the scaling unit 212 determines not to perform chrominance residual scaling (step S4). In this case, the scaling unit 212 and the synthesis unit 213 perform a chrominance block reconstruction process without chrominance residual scaling (step S5).

[0102] Thus, the scaling unit 212 according to the present embodiment performs chrominance residual scaling when the color space transform application flag (cu_act_enabled_flag) is TRUE ("1") regardless of the significant coefficient flags (tu_cb_coded_flag, tu_cr_coded_flag) of the chrominance components. Thereby, chrominance residual scaling is correctly applied and the encoding efficiency can be improved.

[0103] <Other Embodiments> A program for causing a computer to execute each process performed by the encoding device 1 may be provided. Also, a program for causing a computer to execute each process performed by the decoding device 2 may be provided. The program may be recorded on a computer-readable medium. By using a computer-readable medium, it is possible to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a recording medium such as a CD-ROM or a DVD-ROM.

[0104] The circuits that execute each process performed by the encoding device 1 may be integrated, and the encoding device 1 may be configured by a semiconductor integrated circuit (chipset, SoC). The circuits that execute each process performed by the decoding device 2 may be integrated, and the decoding device 2 may be configured by a semiconductor integrated circuit (chipset, SoC).

[0105] As described above, the embodiments have been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist.

Explanation of Signs

[0106] 1: Encoding device 2: Decoding device 100: Block division unit 101: Luminance mapping unit 102: Residual generation unit 103: Scaling unit 104: Color space conversion unit 105: Conversion unit 106: Quantization unit 107: Entropy encoding unit 108: Inverse quantization unit 109: Inverse conversion unit 110: Color space inverse conversion unit 111: Scaling unit 112: Synthesis unit 113: Luminance inverse mapping unit 114: In-loop filter 115: Memory 116: Intra prediction unit 117: Inter prediction unit 118: Luminance mapping unit 119: Switching unit 120: Conversion and quantization unit 130: Inverse quantization and inverse conversion unit 140: Prediction unit 200: Entropy decoding unit 201: Inverse quantization unit 202: Inverse conversion unit 203: Color space inverse conversion unit 204: Synthesis Unit 205: Luminance Inverse Mapping Unit 206: In-Loop Filter 207: Memory 208: Intra Prediction Unit 209: Inter Prediction Unit 210: Luminance Mapping Unit 211: Switching Unit 212: Scaling Unit 213: Synthesis Unit 214: In-Loop Filter 215: Memory 216: Intra Prediction Unit 217: Inter Prediction Unit 218: Switching Unit 220: Inverse Quantization and Inverse Transformation Unit 230: Prediction Unit 240: Prediction Unit

Claims

1. A decoding device that performs a decoding process on blocks obtained by dividing an original image composed of a plurality of color components, an entropy decoding unit that decodes a bitstream to output a transform coefficient for each color component of the block, a first flag indicating, for each color component, whether the block includes a non-zero transform coefficient, and a second flag indicating whether a prediction residual transform process that generates a new prediction residual for each color component by a process using prediction residuals of two or more color components has been applied to the block; an inverse quantization and inverse transform unit that reconstructs the prediction residual from the transform coefficients for each color component; a prediction residual transform processing unit that performs a prediction residual transform processing on the prediction residual when the second flag indicates that the prediction residual transform processing has been applied to the block; a scaling unit that performs chrominance residual scaling by scaling the prediction residual of a chrominance component based on a luminance component corresponding to the chrominance component, The scaling unit determines whether to perform the chrominance residual scaling based on the second flag when the first flag of the chrominance component indicates that the chrominance component includes the zero transform coefficient. Decryption device.

2. A program that causes a computer to function as the decoding device according to claim 1.

3. A decoding method for performing a decoding process on blocks obtained by dividing an original image composed of a plurality of color components, outputting, by decoding the bitstream, transform coefficients for each color component of the block, a first flag indicating for each color component whether the block includes a non-zero transform coefficient, and a second flag indicating whether a prediction residual transform process has been applied to the block, the prediction residual transform process generating a new prediction residual for each color component by a process using prediction residuals of two or more color components; performing a prediction residual transform process on the prediction residual reconstructed from the transform coefficients if the second flag indicates that the prediction residual transform process has been applied to the block; and when the first flag for a chrominance component indicates that the chrominance component includes a zero transform coefficient, determining, based on the second flag, whether to perform chrominance residual scaling, in which the prediction residual for the chrominance component is scaled based on a luminance component corresponding to the chrominance component.

Images