Encoding device, decoding device and storage medium

Abstract

An encoding apparatus encodes each encoding target block. The encoding apparatus includes: a prediction unit that generates a prediction block corresponding to the encoding target block for each component; a residual generation unit that generates a prediction residual for each component, the prediction residual representing the difference between the encoding target block and the prediction block; a color space transformation unit that performs color space transformation processing on the prediction residual for each component; a transformation unit that generates transformation coefficients by performing transformation processing on the prediction residuals; a quantization control unit that determines a scaling list for quantization processing of the transformation coefficients; and a quantization unit that performs the quantization processing on the transformation coefficients using the determined scaling list, the quantization control unit determining the scaling list based on the color space transformation processing.

Description

Technical Field

[0001] This invention relates to encoding devices, decoding devices, and programs. Background Technology

[0002] In the VVC standard scheme, a technique called Adaptive Color Transform (ACT) is used: Given an input image with a 4:4:4 chroma format, after obtaining the difference between the coded object block obtained from segmenting the original image and its prediction block (i.e., the prediction residual), the color space (RGB space) of the prediction residual is transformed to YCgCo space. Then, a transformation process is performed on the prediction residual after the color space transformation. Entropy encoding and other encoding processes (see Non-Patent Literature 1).

[0003] The encoding device can control whether ACT is applied to each encoded object block, and applies a marker to the streaming output ACT for each encoded object block. Therefore, due to ACT, the transformation processing in the RGB space is mixed within the overall image. Quantization is applied to the encoded object block of the prediction residual and transform processing is applied after transforming the prediction residual from RGB space to YCgCo space. Encoded object blocks that have undergone quantization processing.

[0004] However, in VVC, during the quantization of the transform coefficients obtained by transforming the prediction residuals, a scaling list (also called a quantization matrix) is used to control the quantization step size for each frequency component. The scaling list can be set separately for each of the three color components constituting the input image signal (e.g., luminance component, first chromatic aberration component, and second chromatic aberration component). The encoding device sends signals to the decoding side indicating whether a scaling list is applied, and if so, what type of scaling list is applied.

[0005] Existing technical documents

[0006] Non-patent literature

[0007] Non-patent document 1: JVET-Q2001 "Versatile Video Coding (Draft 8)" Summary of the Invention

[0008] The first type of encoding apparatus is characterized by encoding each of a segmented image generated from an image consisting of three or more components including a first component, a second component, and a third component. The encoding apparatus comprises: a prediction unit that generates a prediction block corresponding to the segmented image for each component; a residual generation unit that generates a prediction residual for each component, the prediction residual representing the difference between the segmented image and the prediction block; a color space transformation unit that performs a color space transformation on the prediction residual for each component; a transformation unit that performs a transformation on the prediction residual, thereby generating transformation coefficients; a quantization control unit that determines a scaling list for quantization processing of the transformation coefficients; and a quantization unit that performs the quantization processing on the transformation coefficients using the determined scaling list, wherein the quantization control unit determines the scaling list based on the color space transformation processing.

[0009] The second type of decoding apparatus is characterized by decoding each of a segmented image generated from a decoding object block, the image being composed of three or more components including a first component, a second component, and a third component. The decoding apparatus comprises: an entropy decoding unit that decodes the quantization transform coefficients of the decoding object block from a bitstream for each component; a prediction unit that generates a prediction block corresponding to the decoding object block for each component; an inverse quantization control unit that determines a scaling list for inverse quantization processing; an inverse quantization unit that performs inverse quantization processing on the quantization transform coefficients using the determined scaling list to generate the transform coefficients; an inverse transform unit that performs inverse transform processing on the transform coefficients to generate a prediction residual; a color space inverse transform unit that performs inverse color space transform processing on the prediction residual; and a synthesis unit that synthesizes the prediction residual with the prediction block to generate a decoding block. The inverse quantization control unit determines the scaling list based on the color space inverse transform processing.

[0010] The third type of program enables the computer to function as a first type of encoding device.

[0011] The fourth method of programming enables the computer to function as a decoding device in the second method. Attached Figure Description

[0012] Figure 1 This is a diagram showing the configuration of the encoding device according to an embodiment.

[0013] Figure 2 This diagram illustrates the operation of the quantization control unit and the inverse quantization control unit according to the embodiment.

[0014] Figure 3 This is a diagram showing the configuration of the decoding device according to an embodiment.

[0015] Figure 4 This is a diagram showing the operation of the quantization control unit and the inverse quantization control unit in a modified example.

[0016] Figure 5 This is a diagram showing the bit stream output by the entropy coding section.

[0017] Figure 6 This is a diagram showing the types of NAL units.

[0018] Figure 7 This is a diagram showing the relationship between VPS, SPS, PPS, and APS. Detailed Implementation

[0019] ACT allows control over the color space transformation with or without prediction residuals for each coded object block. Therefore, when encoding an input image as an RGB image, one of two adjacent coded object blocks can be a coded object block with ACT applied, and the other can be a coded object block without ACT applied.

[0020] As shown in Table 1, for coded object blocks applying ACT, after the prediction residuals are transformed from RGB space to YCgCo space, a first scaling list is applied to the prediction residuals of the Y component, a second scaling list is applied to the prediction residuals of the Cg component, and a third scaling list is applied to the prediction residuals of the Co component. On the other hand, for coded object blocks not applying ACT, while maintaining the RGB space for the prediction residuals, a first scaling list is applied to the prediction residuals of the R component, a second scaling list is applied to the prediction residuals of the G component, and a third scaling list is applied to the prediction residuals of the B component.

[0021] [Table 1]

[0022]

[0023] It is known that the majority of the luminance components in the RGB color space are concentrated in the G component, and scaling lists for images in the RGB color space are designed based on the premise that the luminance signal is mainly included in the G component. It is also known that the luminance signal contains more high-frequency components compared to the chrominance signal; therefore, it is highly likely that scaling lists designed for luminance signals will be significantly different in nature from those designed for chrominance signals.

[0024] However, since the first scaling list designed for the R component, which does not include much luminance signal, is applied to the ACT-based transformed Y component by applying the ACT to the prediction residual, it results in the application of a scaling list designed to be applied to components with different properties, which may lead to visual degradation.

[0025] Therefore, the purpose of this disclosure is to suppress image quality degradation even when ACT is applied.

[0026] The encoding and decoding apparatus of the embodiment will be described with reference to the accompanying drawings. The encoding and decoding apparatus of the embodiment respectively encode and decode moving images represented by MPEG (Moving Picture Experts Group). In the following description of the drawings, the same or similar parts are given the same or similar reference numerals.

[0027] <Encoding device>

[0028] First, the configuration of the encoding device in this embodiment will be explained. Figure 1 This is a diagram showing the configuration of the encoding device 1 in this embodiment.

[0029] like Figure 1 As shown, the encoding device 1 includes a block segmentation unit 100, a residual generation unit 110, a switching unit 111, a color space transformation unit 112, a transformation quantization unit 120, a quantization control unit 123, an entropy encoding unit 130, an inverse quantization and inverse transformation unit 140, a synthesis unit 150, a loop filter 160, a memory 170, and a prediction unit 180.

[0030] The block segmentation unit 100 segments the input image, i.e., the original image, which constitutes the frame (or image) unit of the moving image, into multiple image blocks, and outputs the segmented image blocks to the residual generation unit 110. The size of the image block is, for example, 32×32 pixels, 16×16 pixels, 8×8 pixels, or 4×4 pixels. The shape of the image block is not limited to a square, and can also be rectangular (non-square). The image block is the unit for encoding processing by the encoding device 1 (i.e., the encoding target block), and is also the unit for decoding processing by the decoding device (i.e., the decoding target block). Sometimes such an image block is called a CU (Coding Unit).

[0031] In this embodiment, the case where the input image is an RGB signal with a 4:4:4 chroma format will be mainly described. The "R" component corresponds to the first component, the "G" component corresponds to the second component, and the "B" component corresponds to the third component. The block segmentation unit 100 performs block segmentation on the R, G, and B components constituting the image, respectively, and outputs blocks therefrom. In the following description of the encoding apparatus, without distinguishing between the components, they will only be referred to as the encoding target block.

[0032] The residual generation unit 110 calculates the prediction residual, which represents the difference (error) between the encoded target block output by the block segmentation unit 100 and the prediction block obtained by the prediction unit 180 predicting the encoded target block. Specifically, the residual generation unit 110 calculates the prediction residual by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the encoded target block, and outputs the calculated prediction residual to the switching unit 111. In this embodiment, the residual generation unit 110 generates the prediction residual for each component based on the difference between the encoded target block of each component and the prediction block of each component.

[0033] The switching unit 111 outputs the prediction residuals of each component output by the residual generation unit 110 to either the transform quantization unit 120 or the color space transformation unit 112. When no color space transformation processing (ACT) is performed, the switching unit 111 outputs the prediction residuals to the transform quantization unit 120; when color space transformation processing is performed, the switching unit 111 outputs the prediction residuals to the color space transformation unit 112.

[0034] The color space transformation unit 112 performs color space transformation processing on the prediction residuals of each component and outputs the color space transformed prediction residuals to the transformation quantization unit 120. The color space transformation unit 112 generates new prediction residuals by performing the following transformation calculations on the R, G, and B components of the prediction residuals for the encoded object block.

[0035] Co = R - B

[0036] t = B + (Co >> 1)

[0037] Cg = G - t

[0038] Y = t + (Cg >> 1)

[0039] Here, ">>" indicates a right shift operation. Additionally, the "Y" component corresponds to the first component, the "Gg" component to the second component, and the "Co" component to the third component.

[0040] The color space transformation unit 112 can control whether color space transformation processing is performed on each encoded object block. The entropy encoding unit 130 transmits a flag in the bit stream indicating whether color space transformation processing has been performed on the encoded object block.

[0041] Furthermore, the color space transformation processing in the color space transformation unit 112 only requires addition for each component. Subtraction multiplication division The prediction residual, composed of new components, can be generated through shifting and other processes, without necessarily performing a color space transformation. Furthermore, the color space transformation does not need to affect all components. For example, the color space transformation unit 112 could apply a color space transformation such as maintaining the first component, using the average of the second and third components as the new second component, and using the difference between the second and third components as the new third component.

[0042] The transformation and quantization unit 120 performs transformation and quantization processing on a block-by-block basis. The transformation and quantization unit 120 includes a transformation unit 121 and a quantization unit 122.

[0043] The transformation unit 121 performs transformation processing on the prediction residual (referred to as the prediction residual regardless of whether color space transformation processing has been applied) output by the switching unit 111 or the color space transformation unit 112, calculates the transformation coefficients, and the vectorization unit 122 outputs the calculated transformation coefficients. Specifically, the transformation unit 121 generates the transformation coefficients for each component by performing transformation processing on the prediction residual in block units. The transformation processing can be, for example, a frequency transform such as Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), and / or Discrete Wavelet Transform. In addition, the transformation unit 121 outputs information related to the transformation processing to the entropy coding unit 130.

[0044] The transform processing includes transform skipping, which is used in the HEVC (High Efficiency Video Codec) and VVC (Versatile Video Coding) standards, where no transform processing is performed. In HEVC's transform skipping mode, neither horizontal nor vertical transform processing is performed; instead, the prediction residual is scaled and used as transform coefficients. However, the transform skipping in this embodiment also includes transforms that apply only horizontal transform processing and transforms that apply only vertical transform processing. Furthermore, the transform unit 121 can also perform secondary transform processing, further applying transform processing to the transform coefficients obtained through the transform processing. Additionally, the secondary transform processing can be applied only to a portion of the transform coefficients.

[0045] The quantization control unit 123 determines a scaling list (quantization matrix) for quantization processing of the transform coefficients generated by the transform unit 121. Here, a uniform (flat) scaling list is predefined, where all elements of the scaling list have the same value (e.g., all elements have a value of 16). The quantization control unit 123 can set multiple non-uniform scaling lists, each with a different value. Furthermore, when a scaling list is defined with a small size for a large block and then enlarged during actual use, it is not necessary to set different values ​​for all elements. The entropy encoding unit 130 streams information indicating which scaling list was used for quantization processing. Additionally, the non-uniform scaling list is also a scaling list that can have at least some different values ​​set as the values ​​of its elements; it is a variable scaling list.

[0046] In this embodiment, the quantization control unit 123 determines the scaling list based on whether the color space transformation unit 112 performs color space transformation processing. Specifically, when setting a non-uniform scaling list in which the values ​​of the elements of the scaling list are at least partially different, the quantization control unit 123 determines whether to use a non-uniform scaling list or a uniform scaling list based on whether the color space transformation unit 112 performs color space transformation processing (i.e., whether the switching unit 111 outputs the prediction residual to the color space transformation unit 112).

[0047] For example, when the color space transformation unit 112 does not perform color space transformation processing, the quantization control unit 123 determines a non-uniform scaling list as the scaling list used by the quantization unit 122 for quantization processing. On the other hand, when the color space transformation unit 112 performs color space transformation processing, the quantization control unit 123 determines a uniform scaling list as the scaling list used by the quantization unit 122 for quantization processing.

[0048] Here, since a non-uniform scaling list is set for each color component, the properties differ depending on the component being targeted. Therefore, when there are mixed coded object blocks with and without ACT applied within an image, applying the non-uniform scaling list will cause image quality degradation. In this embodiment, when color space transformation processing is performed in the color space transformation unit 112 (i.e., when ACT is applied), image quality degradation can be suppressed by using the uniform scaling list.

[0049] The quantization unit 122 quantizes the transform coefficients output from the transform unit 121 using quantization parameters and a scaling list, and outputs the quantized transform coefficients to the entropy encoding unit 130 and the inverse quantization and inverse transform unit 140. Here, the scaling list for the quantization processing of the quantization unit 122 is determined by the quantization control unit 123. Furthermore, the quantization unit 122 outputs information related to the quantization processing (specifically, information about the quantization parameters and scaling list used in the quantization processing) to the entropy encoding unit 130 and the inverse quantization unit 141.

[0050] Furthermore, the quantization parameter is a parameter that sets a value for a block. Specifically, the quantization parameter is a parameter that is applied to all transform coefficients within the block and determines the quantization coarsness (step size).

[0051] The scaling list is a matrix (quantization matrix) consisting of values ​​set for each component within a block. Specifically, the scaling list consists of values ​​(weighting coefficients) set for each component with i×j elements according to the block size, used to adjust the quantization coarsness for each component of the transform coefficients from low frequency to high frequency. The entropy encoding unit 130 sends the non-uniform scaling list as a signal to the decoding side.

[0052] The entropy encoding unit 130 performs entropy encoding on the quantization transform coefficients output by the quantization unit 122, compresses the data, generates a bitstream (encoded data), and outputs the bitstream to the decoding side. Entropy encoding can use Huffman coding and / or CABAC (Context-based Adaptive Binary Arithmetic Coding), etc. Additionally, the entropy encoding unit 130 includes information related to the transform processing input from the transform unit 121 in the bitstream and sends it as a signal to the decoding side, or includes information related to the prediction processing input from the prediction unit 180 in the bitstream and sends it as a signal to the decoding side. Furthermore, for each encoded object block, the entropy encoding unit 130 includes a marker indicating whether or not ACT color space transformation has been applied and sends it as a signal to the decoding side in the bitstream.

[0053] The inverse quantization and inverse transformation unit 140 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization and inverse transformation unit 140 includes an inverse quantization unit 141 and an inverse transformation unit 142.

[0054] The inverse quantization unit 141 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122. Specifically, the inverse quantization unit 141 performs inverse quantization on the quantization transform coefficients output by the quantization unit 122 using quantization parameters (Qp) and a scaling list to recover the transform coefficients, and outputs the recovered transform coefficients to the inverse transform unit 142. Here, the scaling list for the inverse quantization processing of the inverse quantization unit 141 is determined by the quantization control unit 123.

[0055] The inverse transform unit 142 performs an inverse transform process corresponding to the transform process performed by the transform unit 121, based on the transform category information output by the transform unit 121. For example, if the transform unit 121 performs a discrete cosine transform, the inverse transform unit 142 performs an inverse discrete cosine transform. The inverse transform unit 142 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 141, restores the prediction residual, and outputs the restored prediction residual, i.e., the restored prediction residual, to the synthesis unit 150.

[0056] The synthesis unit 150 synthesizes the restored prediction residual output from the inverse transform unit 142 and the prediction block output from the prediction unit 180 on a pixel-by-pixel basis. The synthesis unit 150 adds each pixel value of the restored prediction residual to each pixel value of the prediction block, decodes (reconstructs) the coded target block, and outputs the decoded block to the loop filter 160. In addition, the decoded block is sometimes also referred to as the reconstructed block.

[0057] The loop filter 160 filters the decoded block output by the synthesis unit 150 and outputs the filtered decoded block to the memory 170.

[0058] The memory 170 stores the filtered decoded blocks output by the loop filter 160, storing the decoded blocks as decoded images in frame units. The memory 170 outputs the stored decoded blocks or decoded images to the prediction unit 180.

[0059] The prediction unit 180 performs prediction processing on a block-by-block basis. The prediction unit 180 generates a prediction block for each component by performing prediction processing such as intra-frame prediction and inter-frame prediction for the coded target block. The prediction unit 180 in this embodiment includes an inter-frame prediction unit 181, an intra-frame prediction unit 182, and a switching unit 183.

[0060] The inter-frame prediction unit 181 performs inter-frame prediction utilizing inter-frame correlation. Specifically, the inter-frame prediction unit 181 uses the decoded image stored in the memory 170 as a reference image, calculates motion vectors using methods such as block matching, predicts the target block to be encoded, generates an inter-frame prediction block, and outputs the generated inter-frame prediction block to the switching unit 183. Here, the inter-frame prediction unit 181 selects the most suitable inter-frame prediction method from inter-frame prediction using multiple reference images (typically, dual prediction) and inter-frame prediction using a single reference image (unidirectional prediction), and performs inter-frame prediction using the selected method. The inter-frame prediction unit 181 outputs information related to inter-frame prediction (motion vectors, etc.) to the entropy coding unit 130.

[0061] The intra-prediction unit 182 performs intra-prediction utilizing intra-frame spatial correlation. Specifically, the intra-prediction unit 182 generates an intra-prediction block by referring to decoded pixels surrounding the coded target block in the decoded image stored in the memory 170, and outputs the generated intra-prediction block to the switching unit 183. The intra-prediction unit 182 selects an intra-prediction mode applicable to the coded target block from a plurality of intra-prediction modes, and uses the selected intra-prediction mode to predict the coded target block.

[0062] The switching unit 183 switches between the inter-frame prediction block output by the inter-frame prediction unit 181 and the intra-frame prediction block output by the intra-frame prediction unit 182, and outputs any prediction block to the residual generation unit 110 and the synthesis unit 150.

[0063] Thus, the encoding apparatus 1 of this embodiment encodes each of the encoding target blocks generated by segmenting an image composed of three or more components including a first component, a second component, and a third component. The encoding apparatus 1 includes: a prediction unit 180 that generates a prediction block corresponding to the encoding target block for each component; a residual generation unit 110 that generates a prediction residual representing the difference between the encoding target block and the prediction block for each component; a color space transformation unit that performs color space transformation processing on the prediction residual; a transformation unit 121 that generates transformation coefficients by performing transformation processing on the prediction residual; a quantization control unit 123 that determines a scaling list for quantization processing on the transformation coefficients; and a quantization unit 122 that performs quantization processing on the transformation coefficients using the determined scaling list.

[0064] Next, the operation of the quantization control unit 123 in this embodiment will be explained. Figure 2 This diagram illustrates the operation of the quantization control unit 123 in this embodiment. The quantization control unit 123 performs quantization control for each encoded object block. Figure 2 The operation. Additionally. Figure 2 The operation is based on the premise of using a non-uniform scaling list, with settings based on factors other than ACT (color space transformation processing).

[0065] like Figure 2 As shown, in step S11, the quantization control unit 123 determines whether to apply ACT (color space transformation processing) to the encoded object block.

[0066] If ACT is not applied to the encoded object block (step S11: No), in step S12, the quantization control unit 123 determines a non-uniform scaling list as a scaling list for the transform processing corresponding to the encoded object block. The non-uniform scaling list can also be a scaling list designed to suit the characteristics of each component in the RGB space. For example, the non-uniform scaling list can be a scaling list designed with the premise that the luminance signal is mainly included in the G component. For example, the quantization control unit 123 applies a first non-uniform scaling list to the prediction residual of the R component, a second non-uniform scaling list to the prediction residual of the G component, and a third non-uniform scaling list to the prediction residual of the B component.

[0067] On the other hand, when ACT is applied to the encoded object block (step S11: Yes), in step S13, the quantization control unit 123 determines a uniform scaling list as a scaling list for the transform processing corresponding to the encoded object block. The uniform scaling list is a pre-set scaling list, that is, a scaling list that is shared by both the encoding and decoding sides. For example, the quantization control unit 123 applies the uniform scaling list to the prediction residuals of the Y component, the Cg component, and the Co component, respectively.

[0068] Thus, when ACT is applied, the encoding apparatus 1 of this embodiment uses a uniform scaling list. As a result, even when encoded object blocks for which ACT is applied and encoded object blocks for which ACT is not applied coexist within an image, image quality degradation can be suppressed.

[0069] <Decoding Device>

[0070] Next, the decoding device of this embodiment will be described, focusing on its differences from the encoding device 1. Figure 3 This is a diagram showing the configuration of the decoding device 2 in this embodiment.

[0071] like Figure 3 As shown, the decoding device 2 includes an entropy decoding unit 200, an inverse quantization and inverse transformation unit 210, an inverse quantization control unit 214, a switching unit 215, a color space inverse transformation unit 216, a synthesis unit 220, a loop filter 230, a memory 240, and a prediction unit 250.

[0072] The entropy decoding unit 200 decodes the encoded data (bitstream) and outputs the quantization transform coefficients corresponding to the decoded target block to the inverse quantization and inverse transform unit 210. Additionally, the entropy decoding unit 200 acquires information related to the transform and quantization processes and outputs this information to the inverse quantization and inverse transform unit 210. Furthermore, the entropy decoding unit 200 acquires information related to the prediction process and outputs this information to the prediction unit 250. The entropy decoding unit 200 acquires the color space transformation marker for each encoded target block and outputs the acquired color space transformation marker to the inverse quantization control unit 214 and the switching unit 215.

[0073] The inverse quantization control unit 214 performs the same operation as the quantization control unit 123 of the encoding device 1 based on the color space transformation marker (see reference). Figure 2 ).

[0074] The inverse quantization and inverse transformation unit 210 performs inverse quantization and inverse transformation processing on a block-by-block basis. The inverse quantization and inverse transformation unit 210 includes an inverse quantization unit 211 and an inverse transformation unit 212.

[0075] The inverse quantization unit 211 performs inverse quantization processing corresponding to the quantization processing performed by the quantization unit 122 of the encoding device 1. The inverse quantization unit 211 performs inverse quantization on the quantization transform coefficients output by the entropy decoding unit 200 using quantization parameters (Qp) and a scaling list to recover the transform coefficients of the decoded target block, and outputs the recovered transform coefficients to the inverse transform unit 212. Here, the scaling list for the inverse quantization processing performed by the inverse quantization unit 211 is determined by the inverse quantization control unit 214.

[0076] The inverse transform unit 212 performs an inverse transform process corresponding to the transform process performed by the transform unit 121 of the encoding device 1. The inverse transform unit 212 performs an inverse transform process on the transform coefficients output by the inverse quantization unit 211, restores the prediction residual, and outputs the restored prediction residual (restored prediction residual) to the switching unit 215.

[0077] The switching unit 215 outputs the prediction residuals of each component output by the inverse color space transformation unit 212 to either the compositing unit 220 or the inverse color space transformation unit 216, based on the color space transformation marker. When inverse color space transformation (ACT) is not performed, the switching unit 215 outputs the prediction residuals to the compositing unit 220; when inverse color space transformation is performed, the switching unit 215 outputs the prediction residuals to the inverse color space transformation unit 216.

[0078] The color space inverse transformation unit 216 performs the inverse processing of the color space transformation process performed by the color space transformation unit 112 of the encoding device 1, that is, the color space inverse transformation process, and outputs the prediction residual after the color space inverse transformation process to the synthesis unit 220. Specifically, the inverse transformation calculation is performed using the Y component, Cg component, and Co component of the restored prediction residual.

[0079] t = Y - (Cg >> 1)

[0080] G = Cg + t

[0081] B = t - (Co >> 1)

[0082] R = Co + B

[0083] The synthesis unit 220 synthesizes the prediction residual output by the switching unit 215 or the color space inverse transformation unit 216 with the prediction block output by the prediction unit 250 in pixel units, thereby decoding (reconstructing) the original block and outputting the decoded block to the loop filter 230.

[0084] The loop filter 230 performs filtering on the decoded block output by the synthesis unit 220 and outputs the filtered decoded block to the memory 240.

[0085] The memory 240 stores the decoded blocks output by the synthesis unit 220, storing the decoded blocks as decoded images in frame units. The memory 240 outputs the decoded blocks or decoded images to the prediction unit 250. In addition, the memory 240 outputs frame-unit decoded images to the outside of the decoding device 2.

[0086] The prediction unit 250 predicts each component on a block-by-block basis. The prediction unit 250 includes an inter-frame prediction unit 251, an intra-frame prediction unit 252, and a switching unit 253.

[0087] The inter-frame prediction unit 251 performs inter-frame prediction that utilizes inter-frame correlation. Specifically, based on inter-frame prediction-related information (e.g., motion vector information) output by the entropy decoding unit 200, the inter-frame prediction unit 251 uses the decoded image stored in the memory 240 as a reference image to predict the encoded target block, generates an inter-frame prediction block, and outputs the generated inter-frame prediction block to the switching unit 253.

[0088] The intra-prediction unit 252 performs intra-prediction that utilizes intra-frame spatial correlation. Specifically, the intra-prediction unit 252 uses an intra-prediction mode based on intra-prediction-related information (e.g., intra-prediction mode information) output by the entropy decoding unit 200, refers to decoded pixels located around the encoded target block in the decoded image stored in the memory 240, generates an intra-prediction block, and outputs the generated intra-prediction block to the switching unit 253.

[0089] The switching unit 253 switches between the inter-frame prediction block output by the inter-frame prediction unit 251 and the intra-frame prediction block output by the intra-frame prediction unit 252, and outputs any type of prediction block to the synthesis unit 220.

[0090] Thus, the decoding apparatus 2 of this embodiment decodes each of the decoding target blocks generated by segmenting an image composed of three or more components including a first component, a second component, and a third component. The decoding apparatus 2 includes: an entropy decoding unit 200, which decodes the quantization transform coefficients of the decoding target block from the bitstream for each component; a prediction unit 250, which generates a prediction block corresponding to the decoding target block for each component; an inverse quantization control unit 214, which determines a scaling list for inverse quantization processing; an inverse quantization unit 211, which generates transform coefficients by performing inverse quantization processing on the quantization transform coefficients using the determined scaling list; an inverse transform unit 212, which generates prediction residuals by performing inverse transform processing on the transform coefficients; a color space inverse transform unit 216, which performs color space inverse transform processing on the prediction residuals; and a synthesis unit 220, which generates a decoding block by synthesizing the prediction residuals and the prediction blocks.

[0091] The inverse quantization control unit 214 determines the scaling list based on the color space inverse transformation processing (color space transformation marker). In this embodiment, if a non-uniform scaling list is set to be used on the encoding side, the inverse quantization control unit 214 determines whether to use a non-uniform scaling list or a uniform scaling list based on whether the color space inverse transformation unit 216 performs color space inverse transformation processing.

[0092] For example, if a non-uniform scaling list is set on the encoding side, and the color space inverse transform unit 216 does not perform color space inverse transform processing, the inverse quantization control unit 214 determines the non-uniform scaling list as the scaling list for inverse quantization processing. If the color space inverse transform unit 216 performs color space inverse transform processing, the inverse quantization control unit 214 determines the uniform scaling list as the scaling list for inverse quantization processing.

[0093] Thus, when ACT is applied, the decoding device 2 of this embodiment uses a uniform scaling list. Therefore, even when ACT-applied encoded object blocks and ACT-free encoded object blocks coexist within an image (picture), image quality degradation can be suppressed.

[0094] <Variation Example>

[0095] Next, variations of the above-described embodiments will be described, focusing on their differences from the embodiments described above.

[0096] In this modified example, when the encoding device 1 is configured to use a non-uniform scaling list, the non-uniform scaling list is used even when ACT (color space transformation processing) is applied. However, the quantization control unit 123 of the encoding device 1 changes the correspondence between the three or more components constituting the image and the scaling lists used for each component based on whether ACT is applied.

[0097] Figure 4 This is a diagram illustrating the operation of the quantization control unit 123 in this modified example.

[0098] like Figure 4 As shown, without performing color space transformation processing (step S21: no), in step S22, the quantization control unit 123 of the encoding device 1 determines a first scaling list for the R component as the first component, a second scaling list for the G component as the second component, and a third scaling list for the B component as the third component (see Table 2).

[0099] [Table 2]

[0100]

[0101] On the other hand, when the color space transformation unit performs color space transformation processing (step S21: Yes), in step S23, the correspondence is changed so that a second scaling list is determined for the Y component as the first component (see Table 3).

[0102] [Table 3]

[0103]

[0104] Since the luminance components in the RGB space are mostly concentrated in the G component, a second scaling list for the G component is applied to the Y component. Therefore, by applying an appropriate scaling list to the Y component, image quality degradation can be suppressed. Furthermore, in the example in Table 3, the correspondence is changed so that a third scaling list is determined for the Cg component (which is the second component), and a first scaling list is determined for the Co component (which is the third component).

[0105] Similarly, when the encoding device 1 is set to use a non-uniform scaling list, the inverse quantization control unit 214 of the decoding device 2 changes the correspondence between the three or more components constituting the image and the scaling list used for each component, depending on whether ACT is applied (i.e., whether the color space inverse transformation unit 216 performs color space inverse transformation processing).

[0106] When the color space inverse transformation unit 216 does not perform color space inverse transformation processing, the inverse quantization control unit 214 determines a first scaling list for the R component (which is the first component), a second scaling list for the G component (which is the second component), and a third scaling list for the B component (which is the third component) (see Table 2). On the other hand, when the color space inverse transformation unit 216 performs color space inverse transformation processing, the inverse quantization control unit 214 determines a second scaling list for the Y component (which is the first component) (see Table 3).

[0107] Furthermore, in this modified example, considering the change in the correspondence between each component and each scaling list, we envision the changed correspondence (i.e., the Y component). Cg components The correspondence between the Co component and the first to third scaling lists is fixed (see Table 3). However, this correspondence can be variably set by sending the correspondence from the encoding device 1 (entropy encoding unit 130) to the decoding device 2 using a signal. For example, the encoding device 1 (entropy encoding unit 130) can represent the changed correspondence (i.e., the Y component). Cg components Information regarding the correspondence between the Co components and the first to third scaling lists is included in the Adaptive Parameter Set (APS) described later and transmitted via signals.

[0108] <Other Implementation Methods>

[0109] The entropy encoding unit 130 of the encoding device 1 may also output a sequence parameter set (SPS) or an adaptive parameter set (APS) that includes information (control flags) indicating whether to perform a scaling list determination operation based on color space transformation processing. The entropy decoding unit 200 of the decoding device 2 may also acquire the SPS or APS that includes information indicating whether to perform a scaling list determination operation based on color space inverse transformation processing.

[0110] Furthermore, the entropy encoding unit 130 of encoding device 1 can also output an SPS or APS containing information (control flags) indicating which operation is being performed in determining the scaling list according to the above-described embodiment and which operation is being performed in determining the scaling list according to Modification 1. The entropy decoding unit 200 of decoding device 2 can also acquire an SPS or APS containing information indicating which operation is being performed in determining the scaling list according to the above-described embodiment and which operation is being performed in determining the scaling list according to Modification 1. The entropy encoding unit 130 of encoding device 1 can send the control flag by signaling depending on whether the chroma format is 4:4:4, or it can be configured to control the signaling of the control flag based on whether ACT can be applied to the sequence in the SPS, etc. Specifically, it can be configured to send the control flag by signaling only when a flag in the SPS indicating that ACT can be applied to the encoded sequence is sent by signaling. Alternatively, it can be configured to control the signaling of the control flag based on a flag in the APS indicating whether the scaling list for the color difference signal is included in the APS. Specifically, it can be configured such that the control flag is sent by signal only if the APS includes a scaling list for the color difference signal.

[0111] Figure 5 This is a diagram showing the bit stream output by the entropy coding unit 130.

[0112] like Figure 5 As shown in (a), the bitstream consists of multiple NAL units and a start code placed at the beginning of each NAL unit. The start code is 4 bytes long and is configured to prevent the generation of 0001 (=0x00000001) within the NAL unit. Figure 5 As shown in (b), each NAL element consists of a NAL element header and a payload.

[0113] Figure 6This is a diagram illustrating the types of NAL units. The type of NAL unit is identified by the `nal_unit_type` within the NAL unit header. NAL unit types are classified into VCL (Video Coding Layer) and non-VCL categories. The VCL category corresponds to the coded bitstream, which includes slices of coded objects (CTUs). The VCL category also corresponds to control information required for decoding, such as VPS (Video Parameter Set), SPS (Sequence Parameter Set), PPS (Picture Parameter Set), and APS (Adaptation Parameter Set). VPS, SPS, PPS, and APS are each transmitted by signals from different NAL units.

[0114] Figure 7 This is a diagram showing the relationship between VPS, SPS, PPS, and APS.

[0115] like Figure 7 As shown, the VPS has its own ID (vps_video_parameter_set_id), which is referenced from the SPS. The VPS stores information related to the overall decoding of the bitstream. For example, the VPS includes information such as the maximum number of layers and the DPB (Decoded Picture Buffer).

[0116] The SPS has its own ID (sps_seq_parameter_set_id), which is referenced from the PPS. Additionally, the SPS has an ID (sps_video_parameter_set_id) that references its own VPS. The SPS stores information needed for sequence decoding. For example, the SPS includes the chroma format and maximum width. Height, bit depth, sub-image information (number, start coordinates of each sub-image, width, height, etc.), and the sequence unit opening of each encoding tool (each function). Information such as on / off controls and / or VUI (Video usability information). On / off in sequence units. The information for the control includes a flag indicating whether the scaling list is applied (sps_scaling_list_enebled_flag).

[0117] The PPS has its own ID (pps_pic_parameter_set_id), referenced from the PH (PictureHeader). Additionally, the PPS has an ID (pps_seq_parameter_set_id) that references its own SPS. The PPS stores information needed for decoding the picture. For example, the PPS includes the image width. Height, block segmentation information (number of blocks in each direction, number of rows) Column width Definitions of height, etc.) and / or slice segmentation information (slice segmentation shape (rectangular / non-rectangular), width of each rectangle in the case of a rectangle). Information such as the number of blocks in the height direction.

[0118] The PH (Header Information) is the header information for each image. The PH is referenced by slices within the image. Since slices implicitly identify themselves within the image, there is no need to define an ID for the PH. On the other hand, the PH maintains the ID (ph_pic_parameter_set_id) of the referenced PPS (Picture Frame Parameter). The PH stores control information for that image. For example, the PH includes the opening parameters for each encoding tool (function) for that image. Information such as on / off controls. PH includes application settings for the tools used by ALF, LMCS, and zoom lists. Information on the switch. (Open / closed by image unit) The information related to the control includes a flag indicating whether the scaling list is applied (ph_scaling_list_present_flag). If more than one tool is applied, the PH includes the ID of the APS that stores the parameter information for that tool.

[0119] APS is a syntax structure used for parameter passing in encoding tools that require passing a large number of parameters such as ALF, LMCS, and scaling lists. APS maintains its own ID, which is referenced from PH.

[0120] By transmitting control information (control flags) of the above-described embodiments and their variations via signals in the SPS, quantization processing can be controlled by a single control flag even when multiple scaling lists are specified, thus reducing the amount of flag information. On the other hand, by transmitting control information (control flags) of the above-described embodiments and their variations via signals in the APS, it is possible to determine whether to control the scaling list applied to each of the multiple defined scaling lists based on the application of color space transformation, thus enabling more flexible quantization processing.

[0121] A program can be provided that enables a computer to perform the processes performed by the encoding device 1 described above. Additionally, a program can be provided that enables a computer to perform the processes performed by the decoding device 2. The program can be stored on a computer-readable medium. If a computer-readable medium is used, the program can be installed on the computer. Here, the computer-readable medium storing the program can be a non-transitory storage medium. There are no particular limitations on the non-transitory storage medium; for example, it can be a CD-ROM, DVD-ROM, or other storage media.

[0122] The circuitry for each process performed by the encoding device 1 can be integrated, and the encoding device 1 can be constructed using a semiconductor integrated circuit (chipset, SoC). Similarly, the circuitry for each process performed by the decoding device 2 can be integrated, and the decoding device 2 can be constructed using a semiconductor integrated circuit (chipset, SoC).

[0123] The embodiments have been described in detail above with reference to the accompanying drawings. However, the specific configuration is not limited to the above configuration, and various design variations can be made without departing from the main idea.

[0124] This application claims priority to Japanese Patent Application No. 2020-070106 (filed on April 8, 2020), the entire contents of which are incorporated herein by reference.

Claims

1. An encoding device, characterized in that, The encoding device encodes each of the encoded object blocks generated from the segmented image, the image being composed of three or more components including a first component, a second component, and a third component. The encoding device includes: The prediction unit generates a prediction block corresponding to the encoded object block for each component; The residual generation unit generates a prediction residual for each component, the prediction residual representing the difference between the encoded object block and the prediction block; The color transformation unit performs color transformation processing on the prediction residual of each component. The color transformation processing is a process of performing arithmetic processing on each component of the prediction residual to generate a new prediction residual composed of new color components. The transformation unit generates transformation coefficients by transforming the predicted residuals. The quantization control unit determines a scaling list for quantization processing of the transform coefficients; The quantization unit performs the quantization process on the transform coefficients using the determined scaling list; as well as The entropy encoding unit outputs a set of sequence parameters including information indicating whether to perform a determination operation on the scaling list based on the color transformation processing.

2. A decoding device, characterized in that, The decoding device decodes each of the decoding object blocks generated from the segmented image, the image being composed of three or more components including a first component, a second component, and a third component. The decoding device includes: The entropy decoding unit decodes the quantization transform coefficients of the decoded object block from the bitstream for each component; The prediction unit generates a prediction block corresponding to the decoded object block for each component; The inverse quantization control unit determines the scaling list used for inverse quantization processing; The inverse quantization unit uses a determined scaling list to perform the inverse quantization process on the quantization transform coefficients to generate the transform coefficients; The inverse transform unit generates the prediction residual by performing an inverse transform on the transform coefficients; The color inverse transformation unit performs color inverse transformation processing on the prediction residual. This color inverse transformation processing involves performing arithmetic operations on each component of the prediction residual to generate a new prediction residual composed of new color components. The synthesis unit generates a decoding block by combining the prediction residual with the prediction block. The entropy decoding unit acquires a sequence parameter set including information indicating whether to perform a determination operation on the scaling list based on the color inverse transformation processing.

3. A stored program storage medium, characterized in that, The program enables the computer to function as the encoding device as described in claim 1.

4. A stored program storage medium, characterized in that, The program enables the computer to function as the decoding device as described in claim 2.

Images