Image processing device and method

Abstract

The present disclosure pertains to an image-processing device and method with which an increase in a delay of an encoding process can be suppressed. When a predetermined condition, which pertains to a quantization parameter used for quantizing coefficient data obtained from coefficient transformation of a prediction residual of an image, is not fulfilled, a quantization coefficient of a color component of the image, which is generated using the quantization parameter, is set to 0. The present disclosure can be applied to, for example, an information-processing device, an encoding device, a decoding device, an electronic apparatus, an information-processing method, a program, etc.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method that are capable of suppressing an increase in delay in encoding processing. [Background technology]

[0002] In a conventional video coding standard, VVC (Versatile Video Coding), a video coding standard, a prediction residual of a video is derived during coding processing, and the prediction residual is subjected to coefficient transformation, quantization, and coding. Also, during decoding processing, coded data is decoded, inversely quantized, and inversely coefficient transformed to derive a prediction residual, and a predicted value is added to the prediction residual to generate (reconstruct) a video (see, for example, Non-Patent Document 1).

[0003] During the inverse quantization of the decoding process, a predicted value is derived using the quantization parameter of the luminance component of the surrounding area, and the quantization parameter of the luminance component used for quantization is derived using the predicted value and the transmitted difference value, and the quantization parameter is applied to the inverse quantization of the luminance component. Furthermore, the quantization parameter of the chrominance component (color component) used for quantization is derived using the quantization parameter of the luminance component processed previously, and the quantization parameter is applied to the inverse quantization of the chrominance component. To enable such processing during the decoding process, quantization parameters are set for the luminance component and the chrominance component during the encoding process, and syntax is generated. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, "Versatile Video Coding Editorial Refinements on Draft 10", JVET-T2001-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 20th Meeting, by teleconference, 7 - 16 Oct. 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] However, it is sometimes difficult to derive the quantization parameters for the chrominance components used for quantization using the quantization parameters for the luminance components, which requires additional processing such as redoing the derivation of the quantization parameters for the luminance components, potentially resulting in delays in the encoding process.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in delay in encoding processing. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present technology is an image processing device including a quantization coefficient update unit that sets, when a predetermined condition related to a quantization parameter used in quantizing coefficient data obtained by coefficient-converting a prediction residual of an image, the quantization coefficient of a color component of the image, generated using the quantization parameter, to 0.

[0008] An image processing method according to one aspect of the present technology is an image processing method in which, when a predetermined condition related to a quantization parameter used in quantizing coefficient data obtained by coefficient-converting a prediction residual of an image is not satisfied, a quantization coefficient of a color component of the image, generated using the quantization parameter, is set to 0.

[0009] In an image processing device and method according to one aspect of the present technology, if a predetermined condition regarding a quantization parameter used to quantize coefficient data obtained by coefficient-converting a prediction residual of an image is not satisfied, the quantization coefficient of a color component of the image generated using the quantization parameter is set to 0. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 10 is a diagram illustrating an example of region division and quantization parameters. [Figure 2] FIG. 10 is a diagram illustrating a quantization group. [Figure 3] FIG. 10 is a diagram illustrating a method for deriving a quantization parameter. [Figure 4] FIG. 10 is a diagram illustrating transmission of prediction residuals of quantization parameters. [Figure 5] FIG. 10 is a diagram illustrating a chroma QP offset. [Figure 6] FIG. 10 is a diagram illustrating a chroma QP offset. [Figure 7] FIG. 10 is a diagram illustrating a chroma QP offset. [Figure 8] FIG. 1 is a diagram illustrating a single tree. [Figure 9] FIG. 10 is a diagram illustrating an example of quantization parameter setting in the case of a single tree. [Figure 10] FIG. 1 is a diagram illustrating a dual tree. [Figure 11] FIG. 10 is a diagram illustrating an example of setting quantization parameters in the case of a dual tree. [Figure 12] 10 is a flowchart illustrating an example of the flow of a quantization process. [Figure 13] FIG. 10 is a diagram illustrating a process for dealing with a constraint violation. [Figure 14] FIG. 1 is a block diagram illustrating an example of the main configuration of an image encoding device. [Figure 15] FIG. 2 is a block diagram illustrating an example of the main configuration of a quantization unit. [Figure 16]10 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 17] 10 is a flowchart illustrating an example of the flow of a quantization process. [Figure 18] FIG. 2 is a block diagram illustrating an example of the main configuration of a quantization unit. [Figure 19] 10 is a flowchart illustrating an example of the flow of a quantization process. [Figure 20] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Quantization parameter constraints 2. Update quantization coefficients 3. First embodiment (image encoding device) 4. Notes

[0012] <1. Quantization parameter constraints> <References supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing, as well as the contents of other documents referenced in the following non-patent documents.

[0013] Non-patent document 1: (mentioned above) Non-patent document 2: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-patent document 3: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0014] In other words, the contents of the above-mentioned non-patent documents are also used as a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly mentioned in the embodiments, they are considered to be within the scope of the disclosure of the present technology and to satisfy the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly mentioned in the embodiments, they are considered to be within the scope of the disclosure of the present technology and to satisfy the support requirements of the claims.

[0015] Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), sub-block, macroblock, tile, or slice, as described in the above-mentioned non-patent document.

[0016] Furthermore, when specifying such block sizes, the block sizes may be specified not only directly but also indirectly. For example, the block sizes may be specified using identification information for identifying the sizes. Furthermore, for example, the block sizes may be specified by the ratio or difference with respect to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. This may reduce the amount of information and improve coding efficiency. Furthermore, the specification of the block sizes may also include specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0017] Furthermore, in this specification, "encoding" refers not only to the overall process of converting an image into a bitstream, but also to some of the processes. For example, it not only includes processes that encompass prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also includes a process that collectively refers to quantization and arithmetic coding, a process that encompasses prediction processing, quantization, and arithmetic coding, etc. Similarly, "decoding" refers not only to the overall process of converting a bitstream into an image, but also to some of the processes. For example, it not only includes processes that encompass inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also includes a process that encompasses inverse arithmetic decoding and inverse quantization, a process that encompasses inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0018] <Quantization parameters> In VVC (Versatile Video Coding) described in Non-Patent Document 1, each picture of a video is divided into regions and processed for each partial region. For example, a picture is divided into CTUs (Coding Tree Units) of a predetermined size. The CTUs are further divided recursively as needed to form CUs (Coding Units).

[0019] Examples of CTUs and CUs are shown in A of Fig. 1. In A of Fig. 1, the rectangular area surrounded by a thick line is a CTU. That is, A of Fig. 1 shows CTU11 and CTU12. CTU12 is further divided as shown by the dotted lines to form CU21 to CU28. Encoding processing is performed for each of these partial areas (CTU and CU).

[0020] Furthermore, the quantization parameters applied to quantization and inverse quantization are set for each partial region of a picture. More specifically, the quantization parameters are set for each rectangular partial region consisting of one or more CUs. For example, for the example of A in FIG. 1, quantization parameters are set for each partial region as shown in B in FIG. 1. In B in FIG. 1, each number in a square indicates an example of a quantization parameter value. That is, for example, a quantization parameter of "30" is set for CTU11. CTU12 is divided into four regions (divided into two vertically and two horizontally), and a quantization parameter of "50" is set for each of regions 31 to 34. Region 31 is made up of CU21. Region 32 is made up of CU22 to CU25. Region 33 is made up of CU26. Region 34 is made up of CU27 and CU28.

[0021] The smallest unit of an area for setting a quantization parameter is called a quantization group. Areas 31 to 34 shown in B of FIG. 1 show examples of this quantization group. The size of the quantization group is determined by parameters (ph_cu_qp_delta_subdiv_intra_slice and ph_cu_qp_delta_subdiv_inter_slice). A and B of FIG. 2 show an example of a portion of the syntax of a picture parameter set (PPS (Picture Parameter Set)). As shown in A of FIG. 2, ph_cu_qp_delta_subdiv_intra_slice, which indicates the minimum size of a quantization group in the case of intra prediction, is defined in the picture parameter set. As shown in B of FIG. 2, ph_cu_qp_delta_subdiv_inter_slice, which indicates the minimum size of a quantization group in the case of inter prediction, is defined in the picture parameter set.

[0022] An example of the semantics of ph_cu_qp_delta_subdiv_intra_slice and ph_cu_qp_delta_subdiv_inter_slice is shown in C of Figure 2. As shown in C of Figure 2, ph_cu_qp_delta_subdiv_intra_slice and ph_cu_qp_delta_subdiv_inter_slice specify the size of the quantization group as the number of divisions from a CTU. For example, when ph_cu_qp_delta_subdiv_intra_slice = 1, the size of one division from a CTU is specified as the size of the quantization group. The same applies to ph_cu_qp_delta_subdiv_inter_slice. For example, as shown in B of Figure 1, when regions 31 to 34 are used as quantization groups, the values ​​of these parameters are set to 1.

[0023] <Quantization parameter signaling> The same quantization parameter is applied to the quantization in the encoding process and the inverse quantization in the decoding process. For this purpose, for example, information about the quantization parameter may be included in a bitstream (encoded data of a moving image) and transmitted from the encoding side to the decoding side (also referred to as signaling). For example, in VVC, the difference value (cu_qp_delta) between the quantization parameter of the luminance component and its predicted value (predQP) is transmitted instead of the quantization parameter itself.

[0024] The quantization parameter Qp is used for quantization and inverse quantization of the luminance component (Y component). Y (or Qp' Y The quantization parameter applied to the quantization and inverse quantization of the Cb component, which is the color difference component (color component), is called qP Cb (or Qp' Cb The quantization parameter applied to the quantization and inverse quantization of the Cr component, which is the color difference component (color component), is called qP Cr (or Qp' Cr The quantization parameter applied to the quantization and inverse quantization of the CbCr components, which are the color difference components (color components), is called qP CbCr (or Qp' CbCr ) is called.

[0025] Quantization parameter Qp of the luminance component Y The predicted value of qP Y_PRED is derived, for example, as follows:

[0026] 1: If any of the following conditions (A1 to A3) is met, the predicted value qP Y_PRED The slice quantization parameter SliceQp Y is set. A1: The processing target is the first quantization group of the slice. A2: The processing target is the first quantization group of the tile. A3: The processing target is the first quantization group in the CTB row of the tile, and entropy_conding_sync_enabled_flag is 1. If none of the conditions A1 to A3 is satisfied, the predicted value qP Y_PRED QP of the previous quantization group in decoding order Y is set.

[0027] 2: If either of the following conditions (B1 and B2) is met, the quantization parameter qP Y_A The predicted value qP Y_PRED is set. B1: availableA (the left block of the quantization group to be processed) is false (FALSE). B2: The left block of the quantization group to be processed straddles the CTU (references outside the CTU). If neither condition B1 nor B2 is satisfied, the quantization parameter qP Y_A quantization parameter QP of (xQg-1, yQg) Y is set.

[0028] 3: If either of the following conditions (C1 and C2) is met, the quantization parameter qP Y_B qP Y_PRED is set. C1:availableB (the upper block of the quantization group to be processed) is false. C2: The block above the quantization group being processed straddles the CTU (references outside the CTU). If neither condition C1 nor C2 is satisfied, the quantization parameter qP Y_B quantization parameter QP of (xQg, yQg-1) Y is set.

[0029] 4: If all of the following conditions (D1 and D2) are met, the predicted value qP Y_PRED QP of (xQg, yQg-1) Y is set. D1: availableB (the upper block of the quantization group to be processed) is true (TRUE). D2: The processing target is the first quantization group in the CTB row of the tile. If neither condition D1 nor D2 is satisfied, the predicted value qP Y_PRED is derived by the following calculation: qP Y_PRED = (qP Y_A + qP Y_B +1)>>1

[0030] In other words, the quantization parameter Qp Y The predicted value of qP Y_PRED is the quantization parameter QP of the surrounding area to be processed. Y Then, as described above, the difference between the quantization parameter and this predicted value is derived and transmitted.

[0031] The quantization parameter (Qp Y (Qp' Y ), qP Cb (Qp' Cb ), qP Cr (Qp' Cr ), qP CbCr (Qp' CbCr An example of a method for deriving the predicted value (qP Y_PRED ) and the transmitted differential value (CuQpDeltaVal) are added to obtain the quantization parameter (Qp Y (Qp' Y ) is derived (reconstructed), and the reconstructed quantization parameters are then applied to the inverse quantization.

[0032] In addition, for the color difference components (Cb component, Cr component, CbCr component), the quantization parameter (Qp Y (Qp' Y )) to calculate the quantization parameter (qP Cb (Qp' Cb ), qP Cr (Qp' Cr ), qP CbCr (Qp' CbCr ) is derived, and the recovered quantization parameters are then applied to the inverse quantization.

[0033] An example of syntax for the transmitted difference value (cu_qp_delta) of the quantization parameter for the luma component is shown in A of Fig. 4. As shown in Fig. 4, the difference value (cu_qp_delta) is transmitted by separating it into an absolute value (cu_qp_delta_abs) and a sign (cu_qp_delta_sign_flag). An example of the semantics of these parameters (cu_qp_delta_abs, cu_qp_delta_sign_flag) is shown in B of Fig. 4.

[0034] <Chroma QP Offset> Note that the quantization parameter for the luma component can be applied as is to the quantization parameter for the chroma component, or an offset can be added to the quantization parameter for the luma component. This offset is also called a chroma QP offset. In this case, the offset value is defined in advance in a list (qp_offset_list) in the picture parameter set, and the offset value to be applied is specified using the index (index) of that list (qp_offset_list).

[0035] An example of this list (qp_offset_list) is shown in Fig. 5. As shown in Fig. 5, this list (qp_offset_list) can store up to six patterns of offset values. The maximum offset value that can be stored is +12, and the minimum is -12. In the example of Fig. 5, the chroma QP offset for the Cb component (Cb_qp_offset_list), the chroma QP offset for the Cr component (Cr_qp_offset_list), and the chroma QP offset for the CbCr components (joint_qp_offset_list) are stored in this list.

[0036] A of Fig. 6 is a diagram showing an example of a portion of the syntax of a picture parameter set. As shown in A of Fig. 6, chroma QP offsets (pps_cb_qp_offset_list[i], pps_cr_qp_offset_list[i], pps_joint_cbcr_qp_offset_list[i]) are set in the picture parameter set. That is, offset values ​​are set in the list (qp_offset_list) for each picture. B of Fig. 6 shows an example of the semantics of the chroma QP offsets (pps_cb_qp_offset_list[i], pps_cr_qp_offset_list[i], pps_joint_cbcr_qp_offset_list[i]).

[0037] When applying a chroma QP offset, by specifying an index (cu_chroma_qp_offset_idx) for each CU, offset values ​​(Cb_qp_offset_list, Cr_qp_offset_list, joint_qp_offset_list) corresponding to the index are specified based on the list (qp_offset_list). A of FIG. 7 is a diagram illustrating an example of a portion of the syntax of a transform unit (transform_unit). As shown in A of FIG. 7, an index (cu_chroma_qp_offset_idx) of the list (qp_offset_list) is specified for each CU. An example of the semantics of cu_chroma_qp_offset_idx is shown in B of FIG. 7.

[0038] <Omission of quantization parameter signaling> Note that for CUs in which the values ​​of all quantization coefficients are 0, inverse quantization is not performed in the decoding process, and therefore transmission of the difference values ​​of the quantization parameters is omitted.

[0039] <Single tree> VVC provides two types of picture region division (also called tree types): single tree and dual tree. In the case of a single tree, the region division method (CU configuration (CU arrangement)) is the same for the luminance component and the chrominance component. An example is shown in FIG. 8. For example, when the luminance component is divided into regions as shown in FIG. 8 and CU21 to CU28 are formed in CTU12 (i.e., when the region division is performed in the same manner as in the example A of FIG. 1), the chrominance component is also divided into regions in the same way. That is, CU21 to CU28 are also formed in CTU12 for the chrominance component.

[0040] Here, it is assumed that the quantization parameters are set in the same manner as in B of Fig. 1. That is, the regions obtained by dividing the CTU into four are set as quantization groups. For example, it is assumed that the quantization parameter "50" is set for each of regions 31 to 34 in CTU12.

[0041] In the case of a single tree, since the CU configuration (CU arrangement) is common to the luminance component and the chrominance component as described above, transmission of the difference value of the quantization parameter is omitted for a CU whose quantization coefficients are all 0 in both the luminance component and the chrominance component (i.e., all quantization coefficients are 0 in all components). In other words, the difference value of the quantization parameter for a CU whose one or more quantization coefficients are not 0 in the luminance component or the chrominance component, or both, are transmitted (i.e., one or more components have non-zero coefficients).

[0042] Furthermore, in the case of a single tree, as described above, the CU configuration (CU arrangement) is common to the luminance component and the chrominance component, so the quantization parameter of each CU of the chrominance component is derived using the quantization parameter of the CU of the luminance component that is in the same position as that CU.

[0043] FIG. 9 shows an example of quantization parameters on the decoding side for CTU12 in the example of FIG. 8. FIG. 9A shows the luminance component (Luma), and FIG. 9B shows the chrominance component (Chroma). In FIG. 9A and FIG. 9B, dotted rounded rectangles indicate quantization groups (i.e., areas 31 to 34). Also, gray CUs indicate CUs whose all quantization coefficients are 0. For example, in FIG. 9A, CU21, CU22, and CU27 all have quantization coefficients of 0. Also, in FIG. 9B, CU23, CU26, and CU27 all have quantization coefficients of 0.

[0044] Of the three rows of values ​​in each CU in A of Figure 9, the value in [ ] in the top row indicates the quantization parameter applied to quantization. The value in the middle row indicates a predicted value of the quantization parameter of that CU derived based on the quantization parameters of surrounding CUs. The value in the bottom row indicates the difference between the quantization parameter and its predicted value (i.e., the value to be transmitted).

[0045] Of the two rows of values ​​in each CU in B of Fig. 9, the values ​​in brackets [ ] in the upper row indicate quantization parameters applied to quantization. The values ​​in the lower row indicate quantization parameters derived based on the quantization parameters of the luma components. For simplicity of explanation, it is assumed here that the quantization parameters of the luma components are directly applied as quantization parameters of the chroma components (no offset is used).

[0046] In the case of CU21, since the quantization coefficients of the chrominance component are not all zero (there are non-zero coefficients), the difference value of the quantization parameter for the luma component is transmitted (+20). During decoding, since the quantization coefficients of the luma component are all zero, inverse quantization of the luma component is omitted. However, since the quantization coefficients of the chroma component are not all zero (there are non-zero coefficients), inverse quantization of the chroma component is performed. In other words, the quantization parameter for the luma component is derived during decoding (inverse quantization). In deriving the quantization parameter for the chroma component, the quantization parameter for the luma component of CU21 is considered to be (50), which is the sum of the transmitted difference value (+20) and the predicted value (30). Therefore, this value (50) is applied as the quantization parameter for the chroma component, and the same value as the quantization parameter (

[50] ) used for quantization is obtained.

[0047] Furthermore, in the case of CU26, the quantization coefficients of the luma component are not all zero (there are non-zero coefficients), so the difference value of the quantization parameter for the luma component is transmitted (+0). During decoding, the quantization coefficients of the luma component are not all zero (there are non-zero coefficients), so inverse quantization of the luma component is performed. A predicted value (50) is derived in deriving the quantization parameter to be applied to the inverse quantization of the luma component. Therefore, the quantization parameter for the luma component is applied as (50), which is the sum of the predicted value (50) and the transmitted difference value (+0). In other words, the same value as the quantization parameter (

[50] ) used for quantization is obtained. For the chroma component, the quantization coefficients are all zero, so inverse quantization is omitted.

[0048] In contrast, for CU27, the quantization coefficients of the luminance component and the chrominance component are all 0. Therefore, the difference value of the quantization parameter is not transmitted (N / A). In this case, inverse quantization of CU27 is omitted for both the luminance component and the chrominance component.

[0049] <In the case of dual trees> In the case of a dual tree, the method of region division (CU configuration (CU arrangement)) is independent for the luminance component and the chrominance component. In other words, the CU configuration for the luminance component and the CU configuration for the chrominance component can be different. An example is shown in FIG. 10. For example, suppose the luminance component (Luma) is divided into regions as shown in example A of FIG. 10, and the chrominance component (Chroma) is divided into regions as shown in example B of FIG. 10. In this case, CU21 to CU28 are formed in CTU12 for the luminance component (i.e., the region division is similar to example A of FIG. 1). In contrast, CU41 to CU44 are formed in CTU12 for the chrominance component (i.e., the region division is different from example A of FIG. 1).

[0050] Here, it is assumed that the quantization parameters are set in the same manner as in B of Fig. 1. That is, the regions obtained by dividing the CTU into four are set as quantization groups. For example, it is assumed that the quantization parameter "50" is set for each of regions 31 to 34 in CTU12.

[0051] In the case of a dual tree, as described above, the CU configuration (CU arrangement) may differ between the luma component and the chroma component, so the quantization parameter is controlled for the luma component, and the processing result of the luma component is used for the chroma component. In other words, the quantization parameter of each CU of the chroma component is derived using the quantization parameter of the luma component CU at the same position as that CU (the luma component CU corresponding to the center position of the chroma component CU).

[0052] Then, transmission of the difference value of the quantization parameter is omitted for a CU in which the quantization coefficients of the luma component are all 0. That is, the value of the quantization coefficient of the chroma component is not taken into consideration in controlling the transmission of the quantization parameter.

[0053] FIG. 11 shows an example of quantization parameters on the decoding side for CTU12 in the example of FIG. 10. FIG. 11A shows the luminance component (Luma), and FIG. 11B shows the chrominance component (Chroma). In FIG. 11A and FIG. 11B, dotted rounded rectangles indicate quantization groups (i.e., areas 31 to 34). Also, gray CUs indicate CUs whose all quantization coefficients are 0. For example, in FIG. 11A, CU21, CU22, and CU27 all have quantization coefficients of 0.

[0054] Of the three rows of values ​​in each CU in A of Figure 11, the value in brackets [ ] in the top row indicates the quantization parameter applied to quantization. The value in the middle row indicates a predicted value of the quantization parameter of that CU derived based on the quantization parameters of surrounding CUs. The value in the bottom row indicates the difference between the quantization parameter and its predicted value (i.e., the value to be transmitted).

[0055] Of the three rows (or two rows) of numerical values ​​in each CU in B of Fig. 11, the numerical value in [ ] in the top row indicates the quantization parameter applied to quantization. Furthermore, the numerical value in the middle row (or the bottom row in the case of two rows) indicates a quantization parameter derived based on the quantization parameter of the luma component. Of the three rows of numerical values ​​in each CU in B of Fig. 11, the numerical value in ( ) in the bottom row indicates an example of a chroma QP offset.

[0056] In the case of CU21 of the luminance component, all quantization coefficients are 0, so transmission of the difference value of the quantization parameter is omitted (N / A). During decoding, all quantization coefficients are 0, so inverse quantization of CU21 (that is, inverse quantization of the luminance component) is omitted.

[0057] However, the quantization coefficients of CU41, the chrominance component corresponding to CU21, are not all zero (there are non-zero coefficients). Therefore, inverse quantization is performed during decoding. That is, during decoding, a quantization parameter for CU41 (chrominance component) is derived. As described above, since the difference value of the quantization parameter for the luma component is not derived, in deriving the quantization parameter for this CU41, the quantization parameter for CU21 is regarded as a predicted value (30) that can be derived from the surrounding CUs. That is, this predicted value (30) is applied as the quantization parameter for inverse quantization of CU41.

[0058] As shown in B of Fig. 11, the quantization parameter used for quantizing CU41 is (

[50] ), which is inconsistent with the derived quantization parameter (30). In other words, in such a case, it is difficult to perform inverse quantization correctly.

[0059] As described above, the quantization parameter of the luma component can be adjusted by applying a chroma QP offset. However, since the range of values ​​is +12 to -12, in the example of CU41 shown in B of Fig. 11, the maximum value of the derived quantization parameter is (42), which does not match the quantization parameter (

[50] ) used for quantization. Therefore, in such a case, it is difficult to perform inverse quantization correctly.

[0060] Such a mismatch in quantization parameters can also occur in CU42 of the color difference components corresponding to CU22 of the luma components.

[0061] <Quantization process flow> In VVC, to avoid such situations, constraints are placed on the quantization parameters of the color difference components during the encoding process, and if these constraints are not respected, i.e., if the situation described above could occur, the quantization parameters are reset.

[0062] An example of the flow of the quantization process in this case will be described with reference to the flowchart in Fig. 12. In this case, first, a quantization parameter QpY for the luma component and a quantization parameter QpC for the chroma component are derived (step S11). Next, using the quantization parameters, the transform coefficients (obtained by performing coefficient transform on the prediction residual of the image) of each CTU are quantized for each of the luma component and the chroma component (step S12). Then, for each CTU of the luma component, syntax elements such as tu_cbf_luma, cu_qp_delta_abs, and cu_qp_delta_sign_flag are generated (step S13).

[0063] Thereafter, it is determined whether the quantization parameters of the chrominance components derived in step S11 comply with the constraints, that is, it is determined whether the quantization parameters of the chrominance components derived in the inverse quantization may be inconsistent with the quantization parameters used in the quantization (step S14).

[0064] If it is determined that the constraints are met, syntax such as tu_cbf_cb and tu_cbf_cr is generated for each CTU of the color difference components (step S15).

[0065] On the other hand, if it is determined in step S11 that the quantization parameters of the chrominance components derived violate the constraints, the process returns to step S11. That is, the process is repeated from step S11 (retry process).

[0066] In this way, if the quantization parameters of the chrominance components violate the constraints, the quantization parameters of the chrominance components must be re-derived (retry processing), which may increase the processing delay of the encoding process.

[0067] <2. Updating quantization coefficients> In such a case, the quantization coefficients are updated so that the quantization parameters of the chrominance components do not violate the constraints. That is, all quantization coefficients of the chrominance components of CUs whose quantization parameters violate the constraints are set to zero.

[0068] For example, in an image processing method, if a predetermined condition regarding a quantization parameter used to quantize coefficient data obtained by coefficient-converting a prediction residual of an image is not satisfied, the quantization coefficient of the color component of the image generated using that quantization parameter is set to 0.

[0069] For example, an image processing device may be provided with a quantization coefficient update unit that sets the quantization coefficient of the color component of an image, generated using a quantization parameter, to 0 when a predetermined condition related to the quantization parameter used to quantize coefficient data obtained by coefficient-converting the prediction residual of the image is not satisfied.

[0070] For example, all quantization coefficients of CU41 and CU42 of the chrominance components shown in B of Fig. 11 are set to 0. An example is shown in A of Fig. 13. In this case, the inverse quantization of CU41 and CU42 is omitted. That is, there is no mismatch between the quantization parameters of the chrominance components derived in the inverse quantization and the quantization parameters of the chrominance components used for quantization. Therefore, the inverse quantization can be performed correctly.

[0071] As described above, by setting all quantization coefficients of the chrominance components of CUs whose quantization parameters violate the constraints to 0, it is possible to prevent mismatches between the quantization parameters of the chrominance components derived in inverse quantization and the quantization parameters of the chrominance components used in quantization. Furthermore, since the retry process (re-deriving the quantization parameters of the chrominance components, etc.) as described above is no longer necessary, it is possible to suppress increases in delay in the encoding process.

[0072] This predetermined condition is arbitrary as long as it can prevent a mismatch between the quantization parameters of the chrominance components derived in the inverse quantization and the quantization parameters of the chrominance components used in the quantization.

[0073] For example, if the tree type of the coefficient data is a dual tree in which the luminance component and the color component of an image are divided into regions independently of each other, and if the predicted value of the quantization parameter of the color component derived based on the luminance component does not match the quantization parameter of the color component applied in quantization, and if the quantization coefficients of the color component generated using the quantization parameter of the color component include a coefficient that is not 0, then all of the quantization coefficients of the color component of that CU may be set to 0. Furthermore, a condition may be added that all of the quantization coefficients of the luminance component CU corresponding to that chrominance component CU are 0.

[0074] <Applying Chroma QP Offset> Furthermore, a chroma QP offset may be applied. That is, if a quantization parameter that matches the quantization parameter of the chroma component used for quantization can be derived by adjustment using the chroma QP offset, application of the chroma QP offset may be prioritized over updating the quantization coefficients described above.

[0075] For example, by using a chroma QP offset, it is possible to determine whether a quantization parameter that matches the quantization parameter of the color difference component used for quantization can be derived, and if it is determined that it can be derived, the chroma QP offset is set, and if it is determined that it cannot be derived, all of the quantization coefficients of that CU can be set to 0.

[0076] For example, as shown in B of Fig. 13, if the quantization parameter of the chrominance component used for quantization is (

[42] ), the quantization parameter (

[42] ) of the chrominance component used for quantization can be obtained using the chroma QP offset (+12) and the predicted value. In such a case, the chroma QP offset is applied, and the update of the quantization coefficient as shown in A of Fig. 13 is omitted.

[0077] By doing so, it is possible to reduce the number of updates to the quantization coefficients. Therefore, it is possible to suppress degradation in the image quality of the decoded image. Furthermore, even when it is impossible to derive a quantization parameter that matches the quantization parameter of the chrominance component used for quantization even after applying a chroma QP offset, it is possible to prevent a mismatch between the quantization parameter of the chrominance component derived in inverse quantization and the quantization parameter of the chrominance component used for quantization. Furthermore, since the above-mentioned retry process (re-deriving the quantization parameter of the chrominance component, etc.) is not necessary, it is possible to suppress an increase in delay in the encoding process.

[0078] 3. First Embodiment <Image encoding device> Fig. 14 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 300 shown in Fig. 14 is a device that encodes image data of a moving image. For example, the image encoding device 300 can encode image data of a moving image using an encoding method described in any of the non-patent documents mentioned above.

[0079] Note that Fig. 14 shows the main processing units (blocks), data flows, etc., and does not necessarily show everything. That is, in the image encoding device 300, there may be processing units that are not shown as blocks in Fig. 14, and there may be processes or data flows that are not shown as arrows, etc. in Fig. 14.

[0080] As shown in FIG. 14 , the image coding device 300 includes a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transformation unit 313, a quantization unit 314, a coding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transformation unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0081] <Control unit> The control unit 301 divides the video data held in the rearrangement buffer 311 into blocks (CUs, PUs, transform blocks, etc.) based on an externally or pre-specified block size of the processing unit. The control unit 301 also determines the coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0082] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 301 supplies them to each block. For example, the header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the coding unit 315 and the prediction unit 322. The transformation information Tinfo is supplied to the coding unit 315, the orthogonal transformation unit 313, the quantization unit 314, the inverse quantization unit 317, and the inverse orthogonal transformation unit 318. The filter information Finfo is supplied to the in-loop filter unit 320.

[0083] <Sorting buffer> Each field (input image) of video data is input to the image coding device 300 in its playback order (display order). The reordering buffer 311 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 301, the reordering buffer 311 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 311 supplies each processed input image to the calculation unit 312. The reordering buffer 311 also supplies each input image (original image) to the prediction unit 322 and the in-loop filter unit 320.

[0084] <Arithmetic section> The calculation unit 312 receives an image I corresponding to a block of processing units and a predicted image P supplied from the prediction unit 322, subtracts the predicted image P from the image I as shown in the following equation, derives a prediction residual D, and supplies it to the orthogonal transformation unit 313.

[0085] D = I - P

[0086] <Orthogonal transformation section> The orthogonal transform unit 313 executes processing related to coefficient transform. For example, the orthogonal transform unit 313 obtains the prediction residual D supplied from the calculation unit 312. The orthogonal transform unit 313 also obtains the transform information Tinfo supplied from the control unit 301.

[0087] The orthogonal transform unit 313 performs an orthogonal transform on the prediction residual D based on the transform information Tinfo, and derives transform coefficients Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate primary transform coefficients. Then, the orthogonal transform unit 313 performs a secondary transform on the primary transform coefficients to generate secondary transform coefficients. The orthogonal transform unit 313 supplies the obtained secondary transform coefficients to the quantization unit 314 as the transform coefficients Coeff.

[0088] Note that the orthogonal transform is an example of coefficient transform, and is not limited to this example. That is, the orthogonal transform unit 313 can perform any coefficient transform on the prediction residual D. Additionally, the orthogonal transform unit 313 can perform any coefficient transform as the primary transform and the secondary transform.

[0089] <Quantization section> The quantization unit 314 performs processing related to quantization. For example, the quantization unit 314 acquires the transform coefficient Coeff supplied from the orthogonal transformation unit 313. The quantization unit 314 also acquires the transform information Tinfo supplied from the control unit 301. The quantization unit 314 then scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. Any quantization method may be used. The quantization rate is controlled by the rate control unit 323. The quantization unit 314 supplies the quantized transform coefficient obtained by this quantization, i.e., the quantized transform coefficient level level, to the encoding unit 315 and the inverse quantization unit 317.

[0090] <Encoding section> The encoding unit 315 executes processing related to encoding. For example, the encoding unit 315 acquires a quantized transform coefficient level "level" supplied from the quantization unit 314. The encoding unit 315 also acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 301. The encoding unit 315 also acquires information related to filters, such as filter coefficients, supplied from the in-loop filter unit 320. The encoding unit 315 also acquires information related to an optimal prediction mode supplied from the prediction unit 322.

[0091] The encoding unit 315 performs variable-length coding (e.g., arithmetic coding) on ​​the quantized transform coefficient level "level" to generate a bit string (encoded data). The encoding unit 315 also derives residual information "Rinfo" from the quantized transform coefficient level "level". The encoding unit 315 then encodes the derived residual information "Rinfo" to generate a bit string.

[0092] The encoding unit 315 includes, in filter information Finfo, information about the filter supplied from the in-loop filter unit 320. Furthermore, the encoding unit 315 includes, in prediction mode information Pinfo, information about the optimal prediction mode supplied from the prediction unit 322. Then, the encoding unit 315 encodes the various encoding parameters described above (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to generate a bit string.

[0093] The encoding unit 315 multiplexes the bit strings of the various types of information generated as described above to generate encoded data, and supplies the encoded data to the accumulation buffer 316.

[0094] <Accumulation buffer> The accumulation buffer 316 temporarily stores the coded data obtained by the coding unit 315. The accumulation buffer 316 outputs the stored coded data, for example, as a bit stream or the like, to the outside of the image coding device 300 at a predetermined timing. For example, this coded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, or the like. In other words, the accumulation buffer 316 also functions as a transmission unit that transmits the coded data (bit stream).

[0095] <Inverse quantization section> The inverse quantization unit 317 performs processing related to inverse quantization. For example, the inverse quantization unit 317 obtains the quantized transform coefficient level "level" supplied from the quantization unit 314. The inverse quantization unit 317 also obtains the transform information "Tinfo" supplied from the control unit 301.

[0096] The inverse quantization unit 317 scales (inverse quantizes) the value of the quantized transform coefficient level level based on the transformation information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed by the quantization unit 314. The inverse quantization unit 317 supplies the transform coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.

[0097] <Inverse orthogonal transform section> The inverse orthogonal transform unit 318 executes processing related to inverse coefficient transform. For example, the inverse orthogonal transform unit 318 acquires the transform coefficient Coeff_IQ supplied from the inverse quantization unit 317. The inverse orthogonal transform unit 318 also acquires the transform information Tinfo supplied from the control unit 301.

[0098] The inverse orthogonal transform unit 318 performs an inverse orthogonal transform on the transform coefficients Coeff_IQ based on the transform information Tinfo, and derives a prediction residual D'. Note that this inverse orthogonal transform is the inverse process of the orthogonal transform performed in the orthogonal transform unit 313. For example, the inverse orthogonal transform unit 318 performs an inverse secondary transform on the transform coefficients Coeff_IQ (secondary transform coefficients) to generate primary transform coefficients. Furthermore, the inverse orthogonal transform unit 318 performs an inverse primary transform on the primary transform coefficients to generate a prediction residual D'. Note that the inverse secondary transform is the inverse process of the secondary transform performed in the orthogonal transform unit 313. Furthermore, the inverse primary transform is the inverse process of the primary transform performed in the orthogonal transform unit 313.

[0099] The inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 319. Note that since the inverse orthogonal transform unit 318 is similar to an inverse orthogonal transform unit (described later) on the decoding side, the description (described later) of the decoding side can be applied to the inverse orthogonal transform unit 318.

[0100] <Arithmetic section> The calculation unit 319 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 318 and the predicted image P supplied from the prediction unit 322. The calculation unit 319 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to derive a locally decoded image Rlocal. The calculation unit 319 supplies the derived locally decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

[0101] <In-loop filter section> The in-loop filter unit 320 executes processing related to in-loop filtering. For example, the in-loop filter unit 320 receives as input a locally decoded image Rlocal supplied from the calculation unit 319, filter information Finfo supplied from the control unit 301, and an input image (original image) supplied from the rearrangement buffer 311. Note that any information may be input to the in-loop filter unit 320, and information other than the above information may also be input. For example, information such as a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, and a block (CU, CTU, etc.) may be input to the in-loop filter unit 320 as needed.

[0102] The in-loop filter unit 320 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses the input image (original image) and other input information for the filtering, as necessary.

[0103] For example, the in-loop filter unit 320 may apply a bilateral filter as the filtering process. Furthermore, the in-loop filter unit 320 may apply a deblocking filter (DBF (DeBlocking Filter)) as the filtering process. Furthermore, the in-loop filter unit 320 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as the filtering process. Furthermore, the in-loop filter unit 320 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as the filtering process. Furthermore, the in-loop filter unit 320 may apply a combination of multiple of these filters as the filtering process. Note that which filters to apply and in what order to apply them are arbitrary and can be selected as appropriate. For example, the in-loop filter unit 320 applies four in-loop filters, namely a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filtering process.

[0104] Of course, the filtering process performed by the in-loop filter unit 320 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 320 may apply a Wiener filter or the like.

[0105] The in-loop filter unit 320 supplies the filtered locally decoded image Rlocal to the frame memory 321. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 320 supplies the information about the filter to the encoding unit 315.

[0106] <Frame memory> The frame memory 321 executes processing related to the storage of image-related data. For example, the frame memory 321 receives as input the locally decoded image Rlocal supplied from the calculation unit 319 and the filtered locally decoded image Rlocal supplied from the in-loop filter unit 320, and holds (stores) them. The frame memory 321 also reconstructs and holds a decoded image R for each picture using the locally decoded image Rlocal (storing it in a buffer within the frame memory 321). The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.

[0107] <Prediction Department> The prediction unit 322 executes processing related to generation of a predicted image. For example, the prediction unit 322 acquires prediction mode information Pinfo supplied from the control unit 301. The prediction unit 322 also acquires an input image (original image) supplied from the rearrangement buffer 311. The prediction unit 322 also acquires a decoded image R (or a part thereof) read from the frame memory 321.

[0108] The prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image). That is, the prediction unit 322 performs prediction and motion compensation by referring to the decoded image R as a reference image, and generates a predicted image P.

[0109] The prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319. Furthermore, the prediction unit 322 supplies information on the prediction mode selected by the above process, i.e., the optimal prediction mode, to the encoding unit 315 as necessary.

[0110] <Rate control section> The rate control unit 323 executes processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of the coded data accumulated in the accumulation buffer 316 so as to prevent overflow or underflow.

[0111] <Quantization coefficient update> The present technology described in <2. Updating Quantization Coefficients> is applied to the image coding device 300 configured as described above. That is, as described above in <2. Updating Quantization Coefficients>, when a predetermined condition related to a quantization parameter used by the quantization unit 314 to quantize coefficient data obtained by coefficient-converting a prediction residual of an image is not satisfied, the quantization coefficient of the color component of the image generated using the quantization parameter is set to 0.

[0112] In this way, the quantization unit 314 can prevent a mismatch between the quantization parameters of the chrominance components derived in the inverse quantization and the quantization parameters of the chrominance components used in the quantization, so as to prevent an increase in delay in the encoding process. In other words, the above-mentioned retry process (re-deriving the quantization parameters of the chrominance components, etc.) is not necessary, and the image encoding device 300 can suppress an increase in delay in the encoding process.

[0113] Furthermore, as described above in <Application of Chroma QP Offset>, if the quantization unit 314 can derive a quantization parameter that matches the quantization parameter of the chroma component used for quantization by adjustment using the chroma QP offset, the quantization unit 314 may prioritize application of the chroma QP offset over updating the above-mentioned quantization coefficient.

[0114] By doing so, it is possible to reduce the number of updates to the quantization coefficients. Therefore, the image encoding device 300 can suppress a decrease in the image quality of the decoded image. Furthermore, even when it is not possible to derive a quantization parameter that matches the quantization parameter of the chrominance component used for quantization even after applying a chroma QP offset, the image encoding device 300 can prevent a mismatch between the quantization parameter of the chrominance component derived in inverse quantization and the quantization parameter of the chrominance component used for quantization. Furthermore, because the above-mentioned retry process (re-deriving the quantization parameter of the chrominance component, etc.) is not necessary, the image encoding device 300 can suppress an increase in delay in the encoding process.

[0115] <Configuration example> These processing units (each processing unit such as the control unit 301 shown in FIG. 14) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Also, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may realize the above-described processing by executing a program using these. Of course, each processing unit may have both of these configurations, and may realize some of the above-described processing by a logic circuit and other by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize some of the above-described processing by a logic circuit, other processing units may realize the above-described processing by executing a program, and still other processing units may realize the above-described processing by both a logic circuit and by executing a program.

[0116] <Quantization section> Fig. 15 is a block diagram showing an example of the main configuration of the quantization unit 314. As shown in Fig. 15, the quantization unit 314 includes a quantization value setting unit 351, a quantization processing unit 352, a luma sine tax generation unit 353, a constraint checking unit 354, a coefficient updating unit 355, and a chroma sine tax generation unit 356.

[0117] The quantization value setting unit 351 acquires the transform coefficients supplied from the orthogonal transform unit 313 (FIG. 14), and sets quantization parameters for each of the luminance component and the chrominance component. The quantization value setting unit 351 supplies the transform coefficients and the quantization parameters to the quantization processing unit 352.

[0118] The quantization processing unit 352 performs quantization on each of the luminance component and the chrominance component using the quantization parameters set by the quantization value setting unit 351, and generates quantized coefficients. The quantization processing unit 352 supplies the quantization parameters and the generated quantized coefficients to the luma sine coefficients generation unit 353.

[0119] The luma syntax generation unit 353 generates syntax related to quantization of the luma component based on any information, such as the quantization coefficients and quantization parameters, supplied from the quantization processing unit 352. For example, the luma syntax generation unit 353 sets parameters such as tu_cbf_luma, cu_qp_delta_abs, and cu_qp_delta_sign_flag. The luma syntax generation unit 353 supplies the generated syntax for the luma component to the constraint checking unit 354, together with the quantization parameters, quantization coefficients, and the like.

[0120] The constraint checking unit 354 determines whether the quantization parameters of the chrominance components violate the constraints (whether the constraints are observed) based on the supplied information. For example, the constraint checking unit 354 determines whether a predetermined condition is satisfied regarding the quantization parameters used for quantizing coefficient data obtained by coefficient-transforming the prediction residual of the image.

[0121] For example, the constraint checking unit 354 determines whether the tree type of the coefficient data is a dual tree in which the luminance component and the color components of an image are divided into regions independently of each other, whether a predicted value of a quantization parameter for a color component derived based on the luminance component does not match the quantization parameter for the color component applied in quantization, and whether the quantized coefficients of the color component generated using the quantization parameter for the color component include a coefficient that is not 0. The constraint checking unit 354 supplies the checking result (determination result) to the coefficient updating unit 355 together with the quantization parameter, the quantized coefficient, the syntax of the luminance component, etc.

[0122] The coefficient updating unit 355 updates the quantization coefficients based on the confirmation result (determination result) by the constraint checking unit 354. That is, when a predetermined condition related to a quantization parameter used for quantizing coefficient data obtained by coefficient-converting the prediction residual of an image is not satisfied, the coefficient updating unit 355 sets the quantization coefficients of the color components of the image generated using the quantization parameter to 0.

[0123] For example, if the tree type of the coefficient data is a dual tree in which the luminance component and the color component of an image are divided into regions independently of each other, and if the predicted value of the quantization parameter of the color component derived based on the luminance component does not match the quantization parameter of the color component applied in quantization, and if the quantization coefficients of the color component generated using the quantization parameter of the color component include a coefficient that is not 0, the coefficient update unit 355 sets all of the quantization coefficients of the color component of that CU to 0.

[0124] When the coefficient updating unit 355 updates the quantization coefficient, it supplies the updated quantization coefficient together with the quantization parameter, the syntax of the luminance component, etc. to the chrominance tax generating unit 356. When the quantization coefficient is not updated, the coefficient updating unit 355 supplies the unupdated quantization coefficient together with the quantization parameter, the syntax of the luminance component, etc. to the chrominance tax generating unit 356.

[0125] The chrominance syntax generator 356 generates syntax related to quantization of the chrominance components based on any information such as the supplied quantization coefficients and quantization parameters. For example, the chrominance syntax generator 356 sets parameters such as tu_cbf_cb and tu_cbf_cr. The chrominance syntax generator 356 supplies the generated syntax for the chrominance components, together with the quantization parameters, quantization coefficients, etc., to the encoding unit 315 and the inverse quantization unit 317 ( FIG. 14 ).

[0126] With this configuration, the quantization unit 314 can prevent a mismatch between the quantization parameters of the chrominance components derived in the inverse quantization and the quantization parameters of the chrominance components used in the quantization, so as to prevent an increase in delay in the encoding process. In other words, the above-mentioned retry process (re-deriving the quantization parameters of the chrominance components, etc.) is not necessary, and the image encoding device 300 can suppress an increase in delay in the encoding process.

[0127] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 300 configured as above will be described with reference to the flowchart of FIG.

[0128] When the image encoding process starts, in step S301, the reordering buffer 311 is controlled by the control unit 301 to reorder the frames of the input video data from display order to encoding order.

[0129] In step S302, the control unit 301 sets a processing unit for the input image held in the sorting buffer 311 (divides the image into blocks).

[0130] In step S303, the control unit 301 determines (sets) coding parameters for the input image held by the reordering buffer 311.

[0131] In step S304, the prediction unit 322 performs a prediction process to generate a predicted image or the like in an optimal prediction mode. For example, in this prediction process, the prediction unit 322 performs intra prediction to generate a predicted image or the like in an optimal intra prediction mode. The prediction unit 322 also performs inter prediction to generate a predicted image or the like in an optimal inter prediction mode. Furthermore, the prediction unit 322 selects an optimal prediction mode from among them based on a cost function value or the like.

[0132] In step S305, the calculation unit 312 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the predicted image. The prediction residual D calculated in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to when the image is encoded as is.

[0133] In step S306, the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated by the process of step S305, and derives a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient. Furthermore, the orthogonal transform unit 313 performs a secondary transform on the primary transform coefficient to generate a secondary transform coefficient (transform coefficient Coeff).

[0134] In step S307, the quantization unit 314 quantizes the transform coefficient Coeff obtained by the processing in step S306, for example, by using the quantization parameter calculated by the control unit 301, and derives the quantized transform coefficient level level.

[0135] In step S308, the inverse quantization unit 317 inverse quantizes the quantized transform coefficient level generated by the process of step S307 using characteristics corresponding to the quantization characteristics of step S307, to derive the transform coefficient Coeff_IQ.

[0136] In step S309, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficients Coeff_IQ obtained by the process of step S308 using a method corresponding to the orthogonal transform process of step S306, thereby deriving prediction residuals D'. For example, the inverse orthogonal transform unit 318 performs inverse secondary transform on the transform coefficients Coeff_IQ (secondary transform coefficients) to generate primary transform coefficients. The inverse orthogonal transform unit 318 also performs inverse primary transform on the primary transform coefficients to generate prediction residuals D'.

[0137] In step S310, the calculation unit 319 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S304 to the prediction residual D' derived in the process of step S309.

[0138] In step S311, the in-loop filter unit 320 performs in-loop filtering on the locally decoded image derived in the process of step S310.

[0139] In step S312, the frame memory 321 stores the locally decoded image derived by the process of step S310 and the locally decoded image filtered in step S311.

[0140] In step S313, the encoding unit 315 encodes the quantized transform coefficient level LEVEL obtained by the process of step S307. For example, the encoding unit 315 encodes the quantized transform coefficient level LEVEL, which is information related to the image, by arithmetic coding or the like to generate encoded data. At this time, the encoding unit 315 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo). Furthermore, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level LEVEL and encodes the residual information RInfo.

[0141] In step S314, the accumulation buffer 316 accumulates the encoded data obtained in this manner and outputs it, for example, as a bit stream, to the outside of the image encoding device 300. This bit stream is transmitted to the decoding side, for example, via a transmission path or a recording medium. In addition, the rate control unit 323 performs rate control as necessary.

[0142] When the process of step S314 ends, the image encoding process ends.

[0143] <Quantization process flow> Next, an example of the flow of the quantization process executed in step S307 in FIG. 16 will be described with reference to the flowchart in FIG.

[0144] When the quantization process starts, in step S351, the quantization value setting unit 351 sets the quantization parameter (Qp Y ) and the quantization parameter of the chrominance component (Qp C ) to set the

[0145] In step S352, the quantization processing unit 352 quantizes the transform coefficients of each CTU for each of the luminance component and the chrominance component, using the quantization parameters set in step S351.

[0146] In step S353, the luma syntax generation unit 353 generates syntax related to quantization for the luma component. For example, the luma syntax generation unit 353 sets parameters such as tu_cbf_luma, cu_qp_delta_abs, and cu_qp_delta_sign_flag.

[0147] In step S354, the constraint checking unit 354 determines whether the quantization parameter of the chrominance component violates a constraint. For example, the constraint checking unit 354 determines whether the tree type of the coefficient data is a dual tree in which the luminance component and the chrominance component of an image are divided into regions independently of each other, whether the predicted value of the quantization parameter of the color component derived based on the luminance component does not match the quantization parameter of the color component applied in quantization, and whether the quantized coefficients of the color component generated using the quantization parameter of the color component include a coefficient that is not 0.

[0148] If it is determined that the constraint is violated, that is, if the tree type of the coefficient data is a dual tree in which the luminance component and the color component of the image are divided into regions independently of each other, the predicted value of the quantization parameter of the color component derived based on the luminance component does not match the quantization parameter of the color component applied in quantization, and it is determined that the quantized coefficient of the color component generated using the quantization parameter of the color component includes a coefficient that is not 0, processing proceeds to step S355.

[0149] In step S355, the coefficient update unit 355 sets the parameters tu_cbf_cb and tu_cbf_cr to 0. tu_cbf_cb is a parameter indicating whether or not the corresponding TU has a non-zero coefficient for the Cb component. When tu_cbf_cb is 0, it indicates that all coefficients of the TU are 0. tu_cbf_cr is a parameter indicating whether or not the corresponding TU has a non-zero coefficient for the Cr component. When tu_cbf_cr is 0, it indicates that all coefficients of the TU are 0.

[0150] In step S356, the coefficient update unit 355 sets all of the coefficients of the CU to be processed to zero.

[0151] When the process of step S356 ends, the process proceeds to step S357. If it is determined in step S354 that the constraints are satisfied, that is, if it is determined that the tree type of the coefficient data is a single tree, that the predicted value of the quantization parameter of the color component derived based on the luminance component matches the quantization parameter of the color component applied in quantization, or that the quantized coefficients of the color component generated using the quantization parameter of the color component do not include a coefficient that is not 0, the process proceeds to step S357.

[0152] In step S357, the chrominance syntax generator 356 generates syntax related to quantization of the chrominance components. For example, the chrominance syntax generator 356 sets parameters such as tu_cbf_cb and tu_cbf_cr.

[0153] When the process of step S357 ends, the quantization process ends and the process returns to FIG.

[0154] By performing each process as described above, the quantization unit 314 can prevent a mismatch between the quantization parameters of the chrominance components derived in the inverse quantization and the quantization parameters of the chrominance components used in the quantization, so as to prevent an increase in delay in the encoding process. In other words, the retry process (re-deriving the quantization parameters of the chrominance components, etc.) as described above is no longer necessary, and the image encoding device 300 can suppress an increase in delay in the encoding process.

[0155] <Quantization section> As described above in <Application of Chroma QP Offset>, if a quantization parameter that matches the quantization parameter of the chroma component used for quantization can be derived by adjustment using the chroma QP offset, application of the chroma QP offset may be prioritized over updating the quantization coefficients described above.

[0156] Fig. 18 is a block diagram showing an example of the main configuration of the quantization unit 314. As shown in Fig. 18, the quantization unit 314 in this case has a chroma QP offset setting unit 371 in addition to the configuration described with reference to Fig. 15.

[0157] In this case, the constraint checking unit 354 further determines whether the quantization parameter used for quantization can be derived using the chroma QP offset. The constraint checking unit 354 supplies the determination result, which is the constraint check result (determination result), to the chroma QP offset setting unit 371 together with the quantization parameter, the quantization coefficient, the syntax of the luminance component, etc.

[0158] The chroma QP offset setting unit 371 acquires various information supplied from the constraint checking unit 354. When the constraint checking unit 354 determines that the quantization parameter of the chroma component violates the constraint and that the quantization parameter used for the quantization can be derived using the chroma QP offset, the chroma QP offset setting unit 371 sets a chroma QP offset for the predicted value in order to derive the quantization parameter used for the quantization.

[0159] Then, the chroma QP offset setting unit 371 supplies the set chroma QP offset and the constraint confirmation result (determination result) together with the quantization parameters, quantization coefficients, syntax of the luminance component, etc. to the coefficient updating unit 355. In this case, the coefficient updating unit 355 supplies the quantization coefficients to the chrominance tax generation unit 356 without updating them.

[0160] Furthermore, if the constraint checking unit 354 determines that the quantization parameters of the chroma components do not violate the constraints, or determines that the quantization parameters used for quantization cannot be derived using the chroma QP offset, the chroma QP offset setting unit 371 omits setting of the chroma QP offset. Then, the chroma QP offset setting unit 371 supplies the information acquired from the constraint checking unit 354 to the coefficient updating unit 355.

[0161] In this case, the coefficient update unit 355 updates the quantization coefficients (sets the value of each coefficient to 0) if the quantization parameters of the chrominance components violate the constraints, and omits updating the quantization coefficients if the quantization parameters of the chrominance components do not violate the constraints.

[0162] With this configuration, it is possible to reduce the number of updates to the quantization coefficients. Therefore, the image encoding device 300 can suppress a decrease in the image quality of the decoded image. Furthermore, even when it is not possible to derive a quantization parameter that matches the quantization parameter of the chrominance component used for quantization even after applying a chroma QP offset, the image encoding device 300 can prevent a mismatch between the quantization parameter of the chrominance component derived in inverse quantization and the quantization parameter of the chrominance component used for quantization. Furthermore, because the above-mentioned retry process (such as re-deriving the quantization parameter of the chrominance component) is not necessary, the image encoding device 300 can suppress an increase in delay in the encoding process.

[0163] <Quantization process flow> An example of the flow of the quantization process executed in step S307 in FIG. 16 in this case will be described with reference to the flowchart in FIG.

[0164] When the quantization process is started, the processes of steps S381 to S384 are executed in the same manner as the processes of steps S351 to S354 in FIG.

[0165] If it is determined in step S384 that the constraints are violated, that is, if the tree type of the coefficient data is a dual tree in which the luminance component and the color components of the image are divided into regions independently of each other, the predicted value of the quantization parameter of the color component derived based on the luminance component does not match the quantization parameter of the color component applied in quantization, and it is determined that the quantized coefficients of the color component generated using the quantization parameter of the color component include a coefficient that is not 0, processing proceeds to step S385.

[0166] In step S385, the constraint checking unit 354 determines whether it is possible to set a chroma QP offset for deriving a quantization parameter for the chrominance component applied to the quantization process. For example, the constraint checking unit 354 determines whether it is possible to set a chroma QP offset for deriving a quantization parameter for the luminance component (qP Y_PRED ) and the chroma QP offset (+ChromaQpOffsetList[idx]) is the quantization parameter Qp of the chroma component applied to the quantization process. C Determine whether it matches (Qp C == qPY_PRED+ChromaQpOffsetList[idx]).

[0167] If it is determined that a chroma QP offset for deriving the quantization parameter of the chrominance component applied to the quantization process can be set, i.e., the sum of the predicted value of the quantization parameter of the luma component and the chrominance QP offset matches the quantization parameter of the chrominance component applied to the quantization process, processing proceeds to step S386.

[0168] In step S386, the chroma QP offset setting unit 371 sets the chroma QP offset. For example, the chroma QP offset setting unit 371 sets cu_chroma_qp_offset_flag, which is flag information indicating whether to apply a chroma QP offset, to true (for example, "1"), and specifies the chroma QP offset using the list index (cu_chroma_qp_offset_idx) (cu_chroma_qp_offset_flag=1, cu_chroma_qp_offset_idx).

[0169] When the process of step S386 ends, the process proceeds to step S389.

[0170] Also, if it is determined in step S385 that the chroma QP offset used to derive the quantization parameter of the chrominance component applied to the quantization process cannot be set, that is, the sum of the predicted value of the quantization parameter of the luma component and the chrominance QP offset does not match the quantization parameter of the chrominance component applied to the quantization process, then processing proceeds to step S387.

[0171] In this case, the processes of steps S387 and S388 are executed in the same manner as the processes of steps S355 and S356 in Fig. 17. When the process of step S388 ends, the process proceeds to step S389.

[0172] Also, if it is determined in step S384 that the constraints are complied with, that is, if the tree type of the coefficient data is a single tree, or the predicted value of the quantization parameter of the color component derived based on the luminance component matches the quantization parameter of the color component applied in quantization, or if it is determined that the quantized coefficients of the color component generated using the quantization parameter of the color component do not include any coefficients other than 0, processing proceeds to step S389.

[0173] The process of step S389 is executed in the same manner as the process of step S357 in Fig. 17. When the process of step S389 ends, the quantization process ends, and the process returns to Fig. 16.

[0174] By performing each process as described above, it is possible to reduce the number of updates to the quantization coefficients. Therefore, the image encoding device 300 can suppress degradation in the image quality of the decoded image. Furthermore, even when it is not possible to derive a quantization parameter that matches the quantization parameter of the chrominance component used for quantization even after applying a chroma QP offset, the image encoding device 300 can prevent a mismatch between the quantization parameter of the chrominance component derived in inverse quantization and the quantization parameter of the chrominance component used for quantization. Furthermore, because the retry process (such as re-deriving the quantization parameter of the chrominance component) described above is unnecessary, the image encoding device 300 can suppress an increase in delay in the encoding process.

[0175] <4. Notes> <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0176] FIG. 20 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0177] In a computer 800 shown in FIG. 20, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0178] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0179] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0180] In the computer configured as above, the CPU 801 executes the above-described series of processes by, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data necessary for the CPU 801 to execute various processes as appropriate.

[0181] The program executed by the computer can be applied by recording it on removable media 821 such as package media, for example. In this case, the program can be installed in storage unit 813 via input / output interface 810 by inserting removable media 821 into drive 815.

[0182] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0183] Alternatively, this program can be installed in advance in the ROM 802 or the storage unit 813 .

[0184] <Applicable targets of this technology> The present technology can be applied to any image encoding method or decoding method. In other words, as long as it does not contradict the above-described present technology, the specifications of various processes related to image encoding, such as coefficient transformation (inverse coefficient transformation), quantization (inverse quantization), encoding, and prediction, are arbitrary and are not limited to the above-described examples. Furthermore, as long as it does not contradict the above-described present technology, some of these processes may be omitted.

[0185] The present technology can also be applied to a multi-viewpoint image coding system that codes a multi-viewpoint image including images from a plurality of views. The present technology can also be applied to a multi-viewpoint image decoding system that decodes coded data of a multi-viewpoint image including images from a plurality of views. In that case, the present technology can be applied in the coding and decoding of each view.

[0186] Furthermore, the present technology can be applied to a hierarchical image coding (scalable coding) system that codes hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. Also, the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes coded data of hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. In this case, the present technology can be applied in the coding and decoding of each layer.

[0187] Additionally, the present technology can be applied to any configuration.

[0188] For example, this technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) used in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communications, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0189] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0190] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0191] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0192] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0193] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0194] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0195] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0196] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0197] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0198] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0199] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0200] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0201] Furthermore, the program executed by the computer may have the following features. For example, the processing of the steps of writing the program may be executed in chronological order according to the order described in this specification. The processing of the steps of writing the program may also be executed in parallel. Furthermore, the processing of the steps of writing the program may be executed individually at the necessary timing, such as when called. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of writing the program may be executed in parallel with the processing of another program. Furthermore, the processing of the steps of writing the program may be executed in combination with the processing of another program.

[0202] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0203] The present technology can also be configured as follows. (1) a quantization coefficient updating unit that, when a predetermined condition regarding a quantization parameter used to quantize coefficient data obtained by coefficient-transforming a prediction residual of an image is not satisfied, sets a quantization coefficient of a color component of the image, generated using the quantization parameter, to 0; An image processing device comprising: (2) The quantization coefficient updating unit sets all of the quantization coefficients of the color components of a CU (Coding Unit) corresponding to the quantization parameter to 0. The image processing device according to (1). (3) further comprising a determination unit that determines whether the condition is satisfied; When the determining unit determines that the condition is not satisfied, the quantization coefficient updating unit sets the quantization coefficient of the color component generated using the quantization parameter to 0. (2) An image processing device according to the present invention. (4) The determination unit the tree type of the coefficient data is a dual tree in which the luminance component and the color component of the image are divided into regions independently of each other; a predicted value of the quantization parameter derived based on the luminance component does not match the quantization parameter; Furthermore, the quantized coefficients of the color components generated using the quantization parameter include coefficients that are not 0. Determine whether (3) An image processing device according to the present invention. (5) The image processing device further includes an offset setting unit that sets an offset to the predicted value in order to derive the quantization parameter. (4) An image processing device according to (4). (6) The determination unit further determines whether the quantization parameter can be derived using the offset; the offset setting unit sets the offset when the determination unit determines that the quantization parameter can be derived; The quantization coefficient updating unit sets the quantization coefficient to 0 when the determining unit determines that the quantization parameter cannot be derived. (5) An image processing device according to (5). (7) A quantization unit that quantizes the coefficient data is further provided. When the determining unit determines that the condition is not satisfied, the quantization coefficient updating unit sets the quantization coefficient of the color component generated by the quantization unit using the quantization parameter to 0. (6) An image processing device according to (6). (8) The image processing apparatus further includes a luma syntax generating unit that generates a syntax related to quantization of the luma component. (7) An image processing device according to (7). (9) The image processing apparatus further includes a chrominance syntax generating unit that generates a syntax related to quantization for the color components. (8) An image processing device according to (8). (10) A quantization parameter setting unit that sets the quantization parameter, The quantization unit quantizes the coefficient data using the quantization parameter set by the quantization parameter setting unit. (9) An image processing device according to (9). (11) The image processing apparatus further includes a coding unit that codes the quantized coefficients. The image processing device according to (10). (12) a storage unit that stores the encoded data generated by the encoding unit; a rate control unit that controls a quantization rate by the quantization unit so that the storage unit does not overflow; The image processing device according to (11) further comprises: (13) The image processing apparatus further includes an orthogonal transform unit that performs an orthogonal transform on the prediction residual to generate the coefficient data. (12) An image processing device according to (12). (14) The image processing device further includes an inverse quantization unit that inversely quantizes the quantized coefficients. (13) An image processing device according to (13). (15) When a predetermined condition regarding a quantization parameter used for quantizing coefficient data obtained by coefficient-transforming a prediction residual of an image is not satisfied, the quantization coefficient of a color component of the image generated using the quantization parameter is set to 0. Image processing methods. [Explanation of symbols]

[0204] 300 image encoding device, 301 control unit, 311 sorting buffer unit, 312 calculation unit, 313 orthogonal transform unit, 314 quantization unit, 315 encoding unit, 316 accumulation buffer unit, 317 inverse quantization unit, 318 inverse orthogonal transform unit, 319 calculation unit, 320 in-loop filter unit, 321 frame memory, 322 prediction unit, 323 rate control unit, 351 quantization value setting unit, 352 quantization processing unit, 353 chrominance tax generation unit, 354 constraint checking unit, 355 coefficient update unit, 356 chrominance tax generation unit, 371 chrominance QP offset setting unit, 800 computer

Claims

1. a quantization coefficient updating unit that sets the quantization coefficients of the color components of the image, which are generated using the quantization parameters, to 0 when a predetermined condition related to the quantization parameters used in quantizing coefficient data obtained by coefficient-transforming the prediction residual of the image is not satisfied. An image processing device comprising:

2. The quantization coefficient updating unit sets all of the quantization coefficients of the color components of a coding unit (CU) corresponding to the quantization parameter to zero. The image processing device according to claim 1 .

3. a determination unit that determines whether the condition is satisfied; When the determining unit determines that the condition is not satisfied, the quantization coefficient updating unit sets the quantization coefficient of the color component generated using the quantization parameter to 0. The image processing device according to claim 2 .

4. The determination unit the tree type of the coefficient data is a dual tree in which the luminance component and the color component of the image are divided into regions independently of each other; a predicted value of the quantization parameter derived based on the luminance component does not match the quantization parameter; Furthermore, the quantized coefficients of the color components generated using the quantization parameter include coefficients that are not zero. Determine whether The image processing device according to claim 3 .

5. The present invention further includes an offset setting unit that sets an offset to the predicted value in order to derive the quantization parameter. The image processing device according to claim 4 .

6. The determination unit further determines whether the quantization parameter can be derived using the offset; the offset setting unit sets the offset when the determination unit determines that the quantization parameter can be derived; The quantization coefficient updating unit sets the quantization coefficient to 0 when the determining unit determines that the quantization parameter cannot be derived. The image processing device according to claim 5 .

7. a quantization unit that quantizes the coefficient data, When the determining unit determines that the condition is not satisfied, the quantization coefficient updating unit sets the quantization coefficient of the color component generated by the quantization unit using the quantization parameter to 0. The image processing device according to claim 6 .

8. The present invention further includes a luma syntax generation unit that generates a syntax related to quantization for the luma component. The image processing device according to claim 7 .

9. The image processing device further includes a chrominance syntax generating unit that generates a syntax related to quantization for the color components. The image processing device according to claim 8 .

10. further comprising a quantization parameter setting unit that sets the quantization parameter, The quantization unit quantizes the coefficient data using the quantization parameter set by the quantization parameter setting unit. The image processing device according to claim 9 .

11. The image processing device further includes a coding unit that codes the quantized coefficients. The image processing device according to claim 10.

12. a storage unit that stores the encoded data generated by the encoding unit; a rate control unit that controls a quantization rate by the quantization unit so that the storage unit does not overflow; The image processing device according to claim 11 , further comprising:

13. an orthogonal transform unit that performs an orthogonal transform on the prediction residual to generate the coefficient data; The image processing device according to claim 12.

14. The image processing device further includes an inverse quantization unit that inversely quantizes the quantized coefficients. The image processing device according to claim 13.

15. If a predetermined condition regarding a quantization parameter used for quantizing coefficient data obtained by coefficient-transforming a prediction residual of an image is not satisfied, the quantization coefficient of the color component of the image generated using the quantization parameter is set to 0. Image processing methods.

Images