Image processing device and method

Abstract

To make it possible to suppress increase in a load of decoding processing.SOLUTION: A secondary transform identifier is set such that secondary transform is performed only in a case where information regarding a block size is equal to or less than a predetermined threshold value, secondary transform is performed on coefficient data derived from image data on the basis of the secondary transform identifier set, and the secondary transform identifier set is encoded and a bitstream is generated. The present disclosure can be applied, for example, to an image processing device, an image encoding device, an image decoding device, an information processing device, an electronic device, an image processing method, an information processing method, and the like.SELECTED DRAWING: Figure 47

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method that can suppress an increase in the load of decoding processing. [Background technology]

[0002] In the past, in image coding, there have been coding tools that perform a low frequency secondary transform (LFST (Low Frequency Secondary Transform)) on transform coefficients after a primary transform, further improving energy compaction (see, for example, Non-Patent Documents 1 and 2). In the low frequency secondary transform, only the coefficient data in the low frequency part of the block to be processed is subjected to secondary transform. Mode information related to this low frequency secondary transform includes a secondary transform identifier st_idx. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, "Versatile Video Coding (Draft 5)", JVET-N1001v8, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 Mar. 2019 [Non-patent document 2] Moonmo Koo, Jaehyun Lim, Mehdi Salehifar, Seung Hwan Kim, "CE6: Reduced Secondary Transform (RST) (CE6-3.1)", JVET-N0193, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 March 2019 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since this secondary transform identifier is determined depending on the total number of non-zero transform coefficients in a CU, it is signaled after each TU in the CU is signaled. Therefore, it is not possible to start inverse quantization or inverse transform processing for each transform block in the CU until decoding of all TUs in the CU and decoding of the secondary transform identifier are completed. This may increase the load of the decoding processing.

[0005] The present disclosure has been made in consideration of such circumstances, and aims to suppress an increase in the load of decoding processing. [Means for solving the problem]

[0006] The image processing device according to one aspect of the present technology is a tree type Only whether it is a single tree or not and a coding unit that arithmetically codes the secondary transformation identifier in accordance with the index of the context set by the setting unit.

[0007] The image processing method according to one aspect of the present technology includes: Only whether it is a single tree or notand arithmetically encoding the secondary transform identifier in accordance with the set index of the context.

[0008] The image processing device according to another aspect of the present technology is a tree type Only whether it is a single tree or not and a decoding unit that arithmetically decodes the secondary transformation identifier according to the index of the context set by the setting unit.

[0009] The image processing method according to another aspect of the present technology is a tree type Only whether it is a single tree or not and arithmetically decoding the secondary transformation identifier in accordance with the set index of the context.

[0010] In the image processing device and method according to one aspect of the present technology, a tree type Only whether it is a single tree or not Based on the above, an index of the context of the secondary transform identifier is set, and the secondary transform identifier is arithmetically coded according to the index of the set context.

[0011] In the image processing device and method according to another aspect of the present technology, a tree type Only whether it is a single tree or not Based on the above, an index of the context of the secondary transform identifier is set, and the secondary transform identifier is arithmetically decoded according to the index of the set context. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 2] FIG. 10 is a diagram illustrating an example of syntax for a transform tree. [Figure 3] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 4] FIG. 10 illustrates an example syntax for counting non-zero transform coefficients. [Figure 5] FIG. 10 is a diagram illustrating an example of a quadtree structure of a CU. [Figure 6] FIG. 10 is a diagram illustrating an example of a flow of decoding a CU. [Figure 7] 10A and 10B are diagrams illustrating an example of how non-zero transform coefficients are counted. [Figure 8] FIG. 10 is a diagram illustrating an example of syntax for setting a context. [Figure 9] FIG. 10 is a diagram illustrating simplification of secondary conversion. [Figure 10] FIG. 10 is a diagram illustrating an example of syntax for deriving the effective transform region size. [Figure 11] FIG. 10 is a diagram illustrating an example of binarization of the last coefficient position. [Figure 12] FIG. 10 is a diagram illustrating an example of a bin sequence at the last coefficient position. [Figure 13] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 14] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 15] FIG. 10 is a diagram illustrating an example of syntax related to transform_mode. [Figure 16] FIG. 10 is a diagram illustrating an example of syntax for setting a context. [Figure 17] FIG. 1 is a block diagram illustrating an example of the main configuration of an image encoding device. [Figure 18] FIG. 2 is a block diagram showing an example of the main configuration of a control unit. [Figure 19] FIG. 2 is a block diagram illustrating an example of the main configuration of an orthogonal transform unit. [Figure 20] 10 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 21] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 22] 10 is a flowchart illustrating an example of the flow of an orthogonal transform process. [Figure 23] FIG. 1 is a block diagram illustrating an example of the main configuration of an image decoding device. [Figure 24] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding unit. [Figure 25] FIG. 2 is a block diagram illustrating an example of the main configuration of an inverse orthogonal transform unit. [Figure 26] 10 is a flowchart showing an example of the flow of an image decoding process. [Figure 27] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 28] 10 is a flowchart illustrating an example of the flow of an inverse orthogonal transform process. [Figure 29] FIG. 10 illustrates an example of syntax for a transform unit. [Figure 30] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 31] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 32] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 33] FIG. 10 illustrates an example of syntax for a transform unit. [Figure 34] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 35] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 36] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 37] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 38] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 39] FIG. 10 is a diagram illustrating an example of syntax for setting a context. [Figure 40] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 41] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 42] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 43] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 44] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 45] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 46] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 47] FIG. 10 is a diagram illustrating an example of syntax related to st_mode. [Figure 48] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 49] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 50] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 51] FIG. 2 is a block diagram showing an example of the main configuration of a control unit. [Figure 52] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 53] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding unit. [Figure 54] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 55] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 56] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 57] FIG. 10 is a diagram illustrating an example of syntax related to residual data within a CU. [Figure 58] 10 is a flowchart showing an example of the flow of an encoding parameter setting process. [Figure 59] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 60] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Secondary Conversion 2. First Embodiment 3. Second Embodiment 4. Third Embodiment 5. Fourth Embodiment 6. Fifth Embodiment 7. Sixth Embodiment 8. Seventh Embodiment 9. Eighth Embodiment 10. Ninth Embodiment 11. Additional Notes

[0014] <1. Secondary Conversion> <1-1. Literature supporting technical content and terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing, as well as the contents of other documents referenced in the following non-patent documents.

[0015] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-Patent Document 4: Recommendation ITU-T H.265 (12 / 2016) "High efficiency video coding", December 2016 Unauthorized Document 5: J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, "Algorithm Description of Joint Exploration Test Model (JEM7)", JVET-G1001, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 7th Meeting: Torino, IT, 13-21 July 2017 Non-licensed Document 6: B. Bross, J. Chen, S. Liu, "Versatile Video Coding (Draft 3)," JVET-L1001, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting: Macau, CN, 3-12 Oct. 2018 Unauthorized Document 7: JJ Chen, Y. Ye, S. Kim, "Algorithm description for Versatile Video Coding and Test Model 3 (VTM 3)", JVET-L1002, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 12th Meeting: Macau, CN, 3-12 Oct. 2018 Non-Patent Document 8: Jianle Chen, Yan Ye, Seung Hwan Kim, "Algorithm description for Versatile Video Coding and Test Model 5 (VTM 5)", JVET-N1002-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 Mar. 2019 Non-Patent Document 9: Mischa Siekmann, Martin Winken, Heiko Schwarz, Detlev Marpe "CE6 - related: Simplification of the Reduced Secondary Transform", JVET-N0555-v3, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 March 2019 Non-Patent Document 10: C. Rosewarne, J. Gan, "CE6-related: RST binarization", JVET-N0105-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 March 2019

[0016] In other words, the contents of the above-mentioned non-patent documents are also used as a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and to meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and to meet the support requirements of the claims.

[0017] Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region of an image (picture) or a processing unit refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" is intended to include any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transform block, sub-block, macroblock, tile, or slice, as described in the above-mentioned non-patent document.

[0018] Furthermore, when specifying such block sizes, the block sizes may be specified not only directly but also indirectly. For example, the block sizes may be specified using identification information for identifying the sizes. Furthermore, for example, the block sizes may be specified by the ratio or difference with respect to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. This may reduce the amount of information and improve coding efficiency. Furthermore, the specification of the block sizes may also include specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0019] Furthermore, in this specification, "encoding" refers not only to the overall process of converting an image into a bitstream, but also to some of the processes. For example, it not only includes processes that encompass prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also includes a process that collectively refers to quantization and arithmetic coding, a process that encompasses prediction processing, quantization, and arithmetic coding, etc. Similarly, "decoding" refers not only to the overall process of converting a bitstream into an image, but also to some of the processes. For example, it not only includes processes that encompass inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also includes a process that encompasses inverse arithmetic decoding and inverse quantization, a process that encompasses inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0020] <1-2. Low-frequency secondary conversion> <Deriving the Secondary Transformation Identifier> In image coding, there is a coding tool that performs a low-frequency secondary transform (LFST (Low Frequency Secondary Transform)) on transform coefficients after a primary transform to further improve energy compaction. In this low-frequency secondary transform, only the coefficient data in the low-frequency part of the block to be processed is subjected to secondary transform. Mode information related to this low-frequency secondary transform includes a secondary transform identifier st_idx.

[0021] 1 shows an example of syntax for residual data (cu_residual) in a CU (coding unit). As shown in this syntax, the secondary transform identifier st_idx is located at the end of the data structure of the CU. That is, st_idx (st_mode) is signaled after each TU (transform_tree) in the CU is signaled.

[0022] FIG. 2 shows an example of syntax related to the transform tree (transform_tree) included in the syntax of FIG. 1. As shown in this syntax, each TU (transform unit) in the CU to be processed is signaled. FIG. 3A shows an example of syntax related to st_mode included in the syntax of FIG. 1. FIG. 3B shows an example of the semantics of sps_st_enabled_flag and st_idx included in this syntax. As shown in the syntax of FIG. 3A, the secondary transform identifier st_idx is signaled. As shown in the semantics of FIG. 3B, the secondary transform identifier st_idx specifies the secondary transform kernel to be applied between two candidate kernels in the selected transform set. st_idx = 0 indicates that no secondary transform is applied.

[0023] The reason why the secondary transform identifier st_idx is signaled after each TU (transform_tree) in the CU in this way is that the condition determination of whether to signal (encode / decode) the secondary transform identifier depends on the total number of non-zero transform coefficients in the regions (also referred to as LFNST corners (or DC sub-blocks)) to which LFNST (Low Frequency Non-Separable Transform) is applied and the regions (also referred to as non-LFNST corners) to which LFNST is not applied in all transform blocks in all TUs included in the CU. In other words, the value of the secondary transform identifier is determined by the number of non-zero transform coefficients in blocks that are subjected to low-frequency secondary transform and the number of non-zero transform coefficients in blocks that are not subjected to low-frequency secondary transform. An example of syntax related to the derivation of these non-zero transform coefficients is shown in Figure 4.

[0024] In such image encoding and decoding, a concept called VPDU (Virtual Pipeline Decording Unit) is applied to enable processing on a 64x64 TU basis. As a result, a 128x128 CU is divided into four 64x64 TUs by implicit TU division (quadtree), as shown in the example of FIG. 5. In the case of a single tree, a TU further includes a TB (transform block) corresponding to component IDs = 0..2 (Y, Cb, Cr). In the case of a luma dual tree, a transform block corresponding to component ID = 0 (Y) is included, and in the case of a chroma dual tree, two transform blocks corresponding to component IDs = 1..2 (Cb, Cr) are included.

[0025] Because of this configuration, the secondary transform identifier is signaled after each TU in the CU is signaled, and therefore, dequantization and inverse transform processing of the first transform block in the CU cannot be started until decoding of all TUs in the CU is completed and further decoding of the secondary transform identifier is completed.

[0026] For example, in the case of a 128x128 CU consisting of TU0 to TU3 as shown in Figure 6, the inverse quantization and inverse transform processing of each transform block in TU0 cannot be started until the decoding of TU0 to TU3 is completed in CABAC and the decoding of the secondary transform identifier st_idx is also completed (i.e., until time T1). In other words, there is a risk of an increase in processing delay.

[0027] 6, in the case of a 128x128 CU, in order to decode the secondary transform identifier st_idx, in the case of a single tree, it was necessary to buffer (store in memory) information (Data1) of transform blocks for 4 TUs x 3 components. In other words, there was a risk that the required memory capacity (i.e., hardware cost) would increase.

[0028] As a result, there is a risk that the load of the decoding process will increase.

[0029] <Block size limit> Therefore, the secondary transformation identifier is signaled only when the information about the block size is equal to or less than a threshold. In other words, the secondary transformation identifier is set so that the secondary transformation is performed only when the information about the block size is equal to or less than a predetermined threshold. In other words, the secondary transformation is performed only on blocks equal to or less than a predetermined size.

[0030] By doing so, it is possible to skip (omit) the secondary transform for coding blocks with a block size larger than a predetermined size. That is, only for coding blocks with a small delay time or memory usage (i.e., coding blocks with a small block size), dequantization and inverse transform processing of the transform block can be started after the secondary transform identifier is decoded, and for coding blocks with a large delay time or memory usage (i.e., coding blocks with a large block size), dequantization and inverse transform processing of the transform block can be started without waiting for the secondary transform identifier to be decoded.

[0031] Therefore, it is possible to suppress an increase in delay and memory usage. That is, it is possible to suppress an increase in the decoding load.

[0032] <st_idx Signaling position> Also, the secondary transform identifier may be signaled outside the footer of the CU. In other words, a secondary transform identifier for setting a secondary transform identifier stored outside the footer of the coded block may be set.

[0033] For example, the secondary transform identifier may be signaled at a position before the footer. For example, in the header of the CU, the secondary transform identifier may be signaled.

[0034] Also, for example, the secondary transform identifier may be signaled in a data unit smaller than the coded block. For example, the secondary transform identifier may be signaled in units of transform units (i.e., common to components). Also, the secondary transform identifier may be signaled in units of transform blocks (i.e., for each component). Further, in units of transform units, the secondary transform identifier for luminance (Y) and the secondary transform identifier for color difference (Cb, Cr) may be signaled respectively.

[0035] By doing so, the buffering period of information required until the inverse quantization and inverse transform processing of the transform block are started can be made shorter than when signaling with the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. That is, it is possible to suppress an increase in the decoding load.

[0036] <Count of non-zero transform coefficients> Furthermore, as shown in Figure 7, to decode the secondary transform identifier st_idx, it is necessary to count the number of non-zero transform coefficients numZeroOutSigCoef in the zero-out region of the LFNST corner in all transform blocks included in the CU and the number of non-zero transform coefficients numSigCoef in all transform blocks included in the CU (Figure 4). Counting the non-zero transform coefficients requires complex processing, which may increase hardware costs. This may increase the load of the decoding process.

[0037] <Omission of counting non-zero transform coefficients> Therefore, the conditional expression that references the number of non-zero transform coefficients is deleted from the decoding / encoding condition of the secondary transform identifier. By doing so, the secondary transform identifier can be derived without the need for complicated processing. Furthermore, when analyzing the secondary transform identifier in the decoding process, the complicated counting of non-zero transform coefficients can be omitted, thereby suppressing an increase in the load of the decoding process.

[0038] <1-3. Context derivation> <ctxinc> Furthermore, the initial context value (offset) ctxInc is derived as shown in the syntax of Fig. 8. That is, ctxInc (mtsCtx) is derived based on the MTS (Multiple Transform Selection) identifier, i.e., the adaptive orthogonal transform identifier tu_mts_idx of the transform unit and the tree type (treeType). Therefore, deriving the context index for the first bin of the secondary transform identifier st_idx requires complex processing, which may increase the load of the decoding process.

[0039] <Simplification of derivation> Therefore, the context is derived without using the adaptive orthogonal transform identifier, which simplifies the derivation of the context and prevents an increase in the load of the decoding process.

[0040] <1-4. Simplifying secondary conversion> <Derivation of effective transformation area size> Furthermore, in the case of the secondary transform described in Non-Patent Document 9, as shown in Fig. 9, primary transform coefficients that are not changed by the RST transform matrix, i.e., primary transform coefficients other than those at the LFNST corners, are zeroed (set to zero), thereby improving coding efficiency.

[0041] For example, within a TB to which a 64x16 RST matrix is ​​applied (i.e., a transform block to which a secondary transform is applied), the area in which valid non-zero transform coefficients may exist is the LFNST corner (DC sub-block) area of ​​4x4 size.

[0042] For example, if the last coefficient position (lastX, lastY) of the DC sub-block is (3, 3), the effective transform area size (log2ZoTbWidth, log2ZoHeight), which is the area where non-zero transform coefficients remain even after zeroing, is derived using a method such as the syntax shown in Figure 10. Then, using this value, each prefix part (last_sig_coeff_x_prefix, last_sig_coeff_y_prefix) of the last coefficient position (lastX, lastY) is binarized according to the table shown in Figure 11. Then, a bin sequence is generated according to the table shown in Figure 12.

[0043] In the case of the 16x16 TB shown in Fig. 9, the bin string bins in the prefix part of the last coefficient position (lastX, lastY) is the code "1110" in the first column from the right and the seventh row from the top in the table shown in Fig. 12. In other words, it is 4 bits.

[0044] If the effective conversion area size is assumed to be 4x4 and binarization is performed, the bin string bins of the prefix part will be the code "111" in the seventh row from the top, second column from the right in the table shown in Fig. 12. That is, there are four bins. In other words, it can be shortened by one bin compared to the above example.

[0045] Considering the X and Y directions, there is room for a maximum reduction of two bins. In other words, the method described in Non-Patent Document 9 may unnecessarily increase the amount of code, resulting in a decrease in coding efficiency. Furthermore, since the amount of code to be decoded increases, the load of the decoding process may increase.

[0046] <Derivation method control based on secondary transformation identifier> Therefore, the effective transform domain size is derived based on the value of the secondary transform identifier. In this way, the effective transform domain size can be derived using a method corresponding to the secondary transform and used to find the prefix part of the last coefficient. This makes it possible to suppress an increase in the code length. In other words, it is possible to suppress an increase in the bin length of the last coefficient (typically, it is possible to reduce the bin length). In other words, it is possible to suppress an increase in the amount of code (suppress a decrease in coding efficiency). As a result, it is possible to suppress an increase in the load of the decoding process.

[0047] 2. First Embodiment <2-1. Secondary Transformation Identifier Signaling> <CUヘッダにおけるシグナリング> The secondary transform identifier st_idx is signaled in the CU header. That is, the secondary transform identifier is signaled before each TU in that CU. For example, during encoding, the secondary transform identifier is set so as to be stored in the header of the coding block. In other words, the secondary transform identifier is set so as to be signaled before each transform block. Also, for example, during decoding, the secondary transform identifier stored in the header of the coding block is analyzed. In other words, the secondary transform identifier signaled before each transform block is analyzed.

[0048] An example of syntax related to the residual data (cu_residual) in a CU in this case is shown in Figure 13. In the example of Figure 13, st_mode (i.e., st_idx) is signaled in the eighth line from the top (gray line). In other words, the secondary transform identifier is signaled before each TU (transform_tree) in the CU (tenth line from the top).

[0049] By doing so, the secondary transform identifier can be decoded before all TUs in the CU are decoded. That is, the inverse quantization and inverse transform processing of each TU can be started before all TUs in the CU are decoded. That is, the buffering period of information required before the inverse quantization and inverse transform processing of the transform block can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. That is, it is possible to suppress an increase in the decoding load.

[0050] <Omission of counting non-zero transform coefficients> In this case, the conditional expression that references the number of non-zero transform coefficients may be deleted from the decoding / encoding condition of the secondary transform identifier. For example, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients in the coding block. Also, for example, the secondary transform identifier may be analyzed regardless of the number of non-zero transform coefficients in the coding block.

[0051] In this way, when deriving a secondary transform identifier in the encoding process, the cumbersome counting of non-zero transform coefficients can be omitted, thereby suppressing an increase in the load of the encoding process.Furthermore, when analyzing a secondary transform identifier in the decoding process, the cumbersome counting of non-zero transform coefficients can be omitted, thereby suppressing an increase in the load of the decoding process.

[0052] <Setting the secondary transform identifier based on the residual PCM mode> There is a residual PCM coding mode (BDPCM (Block-based Delta Pulse Code Modulation)). In this residual PCM coding mode, the difference between the residual of a block for which secondary transform has been skipped and the residual of its adjacent block is derived, and the residual difference for each block is coded. bdpcm_flag is flag information that determines whether or not such a residual PCM coding mode is applied. If bdpcm_flag = 1, the residual PCM coding mode is applied, and if bdpcm_flag = 0, the residual PCM coding mode is not applied.

[0053] In this way, the secondary transform identifier may be signaled only when the residual PCM coding mode is not applied (ie, the secondary transform identifier is not signaled when the residual PCM coding mode is applied).

[0054] This allows the secondary conversion identifier to be easily set according to the residual PCM coding mode. Furthermore, during decoding, the secondary conversion identifier can be easily analyzed according to the residual PCM coding mode. This reduces the load of the decoding process.

[0055] An example of the syntax for st_mode in this case is shown in Figure 14. In this case, as shown in the second row from the top (gray row), bdpcm_flag = 0 is required for stAllowed to be true. In other words, when the residual PCM coding mode is applied, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients, the secondary transform identifier may be set based on a flag indicating whether the residual PCM coding mode is applied.

[0056] Of course, the condition when this bdpcm_flag is not applied (bdpcm_flag == 0) may be excluded from the determination of whether or not to perform secondary conversion (deriving stAllowed).

[0057] <Transform skip and adaptive orthogonal transform settings> Furthermore, in this case, whether or not to skip the conversion may be set based on whether or not the secondary conversion is performed. In other words, the conversion may be skipped only when the secondary conversion is not performed.

[0058] An example of the syntax related to transform_mode in this case is shown in FIG. 15. As shown in FIG. 15, the value of tsAllowed is set in the second row from the top (the row with gray background). For tsAllowed to be true, st_idx = 0 is required. In other words, tsAllowed can be true only when secondary transform is skipped. tsAllowed is a parameter that sets the value of the transform skip flag transform_skip_flag, which indicates whether or not to perform a transform skip that skips (omits) the orthogonal transform process. As shown in the fourth to sixth rows from the top, when tsAllowed is true, the transform skip flag transform_skip_flag is signaled. In other words, when tsAllowed is false, the transform skip flag transform_skip_flag is not signaled. In this case, on the decoding side, it is estimated that transform_skip_flag = 0, and no transform skip is performed. In other words, transform skip can be performed only when secondary transform is skipped.

[0059] In this way, the conversion skip flag can be easily set according to the secondary conversion identifier. Also, during decoding, the conversion skip flag can be easily analyzed according to the secondary conversion identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0060] Of course, skipping the secondary conversion does not have to be included in the conditions for whether or not to skip conversion.

[0061] As with the transform skip flag transform_skip_flag, skipping of secondary transform may be added to the conditions for determining whether or not to apply adaptive orthogonal transform.

[0062] The value of mtsAllowed is set in the third row from the top in Figure 15. As in the case of the transform skip flag transform_skip_flag in Figure 15, st_idx = 0 (&& (st_idx[x0][y0] == 0)) may be added to the setting conditions of this mtsAllowed. In other words, mtsAllowed may be set to true only when secondary transform is skipped. mtsAllowed is a parameter that sets the value of the adaptive orthogonal transform identifier tu_mts_idx, which indicates whether or not to apply adaptive orthogonal transform processing for each TU. As shown in the seventh to ninth rows from the top, when mtsAllowed is true, the adaptive orthogonal transform identifier tu_mts_idx is signaled. In other words, when mtsAllowed is false, the adaptive orthogonal transform identifier tu_mts_idx is not signaled. In this case, on the decoding side, it is estimated that tu_mts_idx = 0, and adaptive orthogonal transform is not applied. That is, adaptive orthogonal transform can be performed only if the secondary transform is skipped.

[0063] In this way, it is possible to easily set an adaptive orthogonal transform identifier according to the secondary transform identifier. Furthermore, during decoding, it is possible to easily analyze the adaptive orthogonal transform identifier according to the secondary transform identifier. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0064] Of course, skipping the secondary transform does not have to be added to the conditions for whether or not to apply the adaptive orthogonal transform. That is, in the syntax of Fig. 15, st_idx = 0(&& (st_idx[x0][y0] == 0)) may be added only to the second row from the top, only to the third row from the top, or to both the second and third rows from the top, or to neither.

[0065] <Context derivation> In this case, for example, as shown in FIG. 16, the initial value (offset) ctxInc of the context may be derived based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx.

[0066] In Fig. 16, mtsCtx is set based only on whether the tree type is a single tree. Therefore, the context index of the first bin of the secondary transformation identifier st_idx can be derived (more easily) without the need for complicated processing. Similarly, during decoding, the context index of the first bin of the secondary transformation identifier st_idx can be derived more easily. Therefore, an increase in the load of the decoding process can be suppressed.

[0067] <2-2. Encoding side> <Image encoding device> Next, the encoding side will be described. Fig. 17 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 100 shown in Fig. 17 is a device that encodes image data of a moving image. For example, the image encoding device 100 encodes image data of a moving image using an encoding method described in any of Non-Patent Documents 1 to 10.

[0068] Note that Fig. 17 shows the main processing units (blocks), data flows, etc., and does not necessarily show everything. That is, in the image encoding device 100, there may be processing units that are not shown as blocks in Fig. 17, and there may be processes and data flows that are not shown as arrows, etc. in Fig. 17.

[0069] As shown in FIG. 17 , the image coding device 100 includes a control unit 101, a rearrangement buffer 111, a calculation unit 112, an orthogonal transformation unit 113, a quantization unit 114, a coding unit 115, an accumulation buffer 116, an inverse quantization unit 117, an inverse orthogonal transformation unit 118, a calculation unit 119, an in-loop filter unit 120, a frame memory 121, a prediction unit 122, and a rate control unit 123.

[0070] <Control unit> The control unit 101 divides the video data held in the rearrangement buffer 111 into blocks (CUs, PUs, transform blocks, etc.) that are processing units based on an externally or pre-specified block size. The control unit 101 also determines coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0071] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 101 supplies them to each block. Specifically, this is as follows.

[0072] Header information Hinfo is provided for each block. The prediction mode information Pinfo is supplied to the encoding unit 115 and the prediction unit 122. The transformation information Tinfo is supplied to the encoding unit 115 , the orthogonal transformation unit 113 , the quantization unit 114 , the inverse quantization unit 117 , and the inverse orthogonal transformation unit 118 . The filter information Finfo is supplied to the in-loop filter unit 120 .

[0073] <Sorting buffer> Each field (input image) of video data is input to the image coding device 100 in its playback order (display order). The reordering buffer 111 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 101, the reordering buffer 111 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 111 supplies each processed input image to the calculation unit 112. The reordering buffer 111 also supplies each input image (original image) to the prediction unit 122 and the in-loop filter unit 120.

[0074] <Arithmetic section> The calculation unit 112 receives an image I corresponding to a block of processing units and a predicted image P supplied from the prediction unit 122, subtracts the predicted image P from the image I as shown in the following equation, derives a prediction residual D, and supplies it to the orthogonal transformation unit 113.

[0075] D = I - P

[0076] <Orthogonal transformation section> The orthogonal transform unit 113 receives as input the prediction residual D supplied from the calculation unit 112 and the transformation information Tinfo supplied from the control unit 101, and performs an orthogonal transform on the prediction residual D based on the transformation information Tinfo to derive the transformation coefficient Coeff. Note that the orthogonal transform unit 113 can perform adaptive orthogonal transform that adaptively selects the type of orthogonal transform (transform coefficient). The orthogonal transform unit 113 supplies the obtained transformation coefficient Coeff to the quantization unit 114.

[0077] <Quantization part> The quantization unit 114 receives as input the transform coefficient Coeff supplied from the orthogonal transformation unit 113 and the transform information Tinfo supplied from the control unit 101, and scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. The quantization rate is controlled by the rate control unit 123. The quantization unit 114 supplies the quantized transform coefficient obtained by this quantization, i.e., the quantized transform coefficient level, to the encoding unit 115 and the inverse quantization unit 117.

[0078] <Encoding part> The encoding unit 115 receives as input the quantized transform coefficient level LEVEL supplied from the quantization unit 114, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 101, information on filters such as filter coefficients supplied from the in-loop filter unit 120, and information on an optimal prediction mode supplied from the prediction unit 122. The encoding unit 115 performs variable-length coding (for example, arithmetic coding) on ​​the quantized transform coefficient level LEVEL to generate a bit string (encoded data).

[0079] Furthermore, the encoding unit 115 derives residual information Rinfo from the quantized transform coefficient level level, encodes the residual information Rinfo, and generates a bit string.

[0080] Furthermore, the encoding unit 115 includes information about the filter supplied from the in-loop filter unit 120 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 122 in the prediction mode information Pinfo. Then, the encoding unit 115 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0081] The encoding unit 115 multiplexes the bit strings of the various types of information generated as described above to generate encoded data, and supplies the encoded data to the accumulation buffer 116.

[0082] <Accumulation buffer> The accumulation buffer 116 temporarily stores the coded data obtained by the coding unit 115. The accumulation buffer 116 outputs the stored coded data, for example, as a bit stream or the like, to the outside of the image coding device 100 at a predetermined timing. For example, this coded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, or the like. In other words, the accumulation buffer 116 also functions as a transmission unit that transmits the coded data (bit stream).

[0083] <Inverse quantization section> The inverse quantization unit 117 performs processing related to inverse quantization. For example, the inverse quantization unit 117 receives the quantized transform coefficient level level supplied from the quantization unit 114 and the transformation information Tinfo supplied from the control unit 101 as input, and scales (inverse quantizes) the value of the quantized transform coefficient level level based on the transformation information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed in the quantization unit 114. The inverse quantization unit 117 supplies the transformation coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 118.

[0084] <Inverse orthogonal transform section> The inverse orthogonal transform unit 118 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 118 receives as input the transform coefficients Coeff_IQ supplied from the inverse quantization unit 117 and the transform information Tinfo supplied from the control unit 101, and performs inverse orthogonal transform on the transform coefficients Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. Note that this inverse orthogonal transform is the inverse process of the orthogonal transform performed in the orthogonal transform unit 113. In other words, the inverse orthogonal transform unit 118 can perform adaptive inverse orthogonal transform that adaptively selects the type of inverse orthogonal transform (transform coefficients).

[0085] The inverse orthogonal transform unit 118 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 119. Note that the inverse orthogonal transform unit 118 is similar to an inverse orthogonal transform unit (described later) on the decoding side, and therefore the description (described later) on the decoding side can be applied to the inverse orthogonal transform unit 118.

[0086] <Arithmetic section> The calculation unit 119 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 118 and the predicted image P supplied from the prediction unit 122. The calculation unit 119 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to derive a locally decoded image Rlocal. The calculation unit 119 supplies the derived locally decoded image Rlocal to the in-loop filter unit 120 and the frame memory 121.

[0087] <In-loop filter section> The in-loop filter unit 120 performs processing related to in-loop filtering. For example, the in-loop filter unit 120 receives as input a locally decoded image Rlocal supplied from the calculation unit 119, filter information Finfo supplied from the control unit 101, and an input image (original image) supplied from the rearrangement buffer 111. Note that any information may be input to the in-loop filter unit 120, and information other than the above information may also be input. For example, information such as a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, and a block (CU, CTU, etc.) may be input to the in-loop filter unit 120 as needed.

[0088] The in-loop filter unit 120 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 120 also uses the input image (original image) and other input information for the filtering, as necessary.

[0089] For example, the in-loop filter unit 120 applies four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate.

[0090] Of course, the filtering process performed by the in-loop filter unit 120 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 120 may apply a Wiener filter or the like.

[0091] The in-loop filter unit 120 supplies the filtered locally decoded image Rlocal to the frame memory 121. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 120 supplies the information about the filter to the encoding unit 115.

[0092] <Frame memory> The frame memory 121 performs processing related to the storage of image-related data. For example, the frame memory 121 receives as input the locally decoded image Rlocal supplied from the calculation unit 119 and the filtered locally decoded image Rlocal supplied from the in-loop filter unit 120, and holds (stores) them. The frame memory 121 also reconstructs and holds a decoded image R for each picture using the locally decoded image Rlocal (storing it in a buffer within the frame memory 121). The frame memory 121 supplies the decoded image R (or a part thereof) to the prediction unit 122 in response to a request from the prediction unit 122.

[0093] <Prediction Department> The prediction unit 122 performs processing related to the generation of a predicted image. For example, the prediction unit 122 receives as input prediction mode information Pinfo supplied from the control unit 101, an input image (original image) supplied from the rearrangement buffer 111, and a decoded image R (or a part thereof) read from the frame memory 121. The prediction unit 122 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction by referring to the decoded image R as a reference image, and performs motion compensation processing based on the prediction result to generate a predicted image P. The prediction unit 122 supplies the generated predicted image P to the calculation unit 112 and the calculation unit 119. Furthermore, the prediction unit 122 supplies information on the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 115 as necessary.

[0094] <Rate control section> The rate control unit 123 performs processing related to rate control. For example, the rate control unit 123 controls the rate of the quantization operation of the quantization unit 114 based on the code amount of the coded data accumulated in the accumulation buffer 116 so as to prevent overflow or underflow.

[0095] These processing units (the control unit 101, the reordering buffer 111, the rate control unit 123, and the like) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0096] The image coding device 100 codes images by applying the various techniques described in <2-1. Signaling of Secondary Translation Identifier>. Therefore, as described above, it is possible to suppress an increase in the load of the decoding process.

[0097] <Control unit> Fig. 18 is a block diagram showing an example of the main configuration of the control unit 101. As shown in Fig. 18, the control unit 101 has a secondary transform identifier setting unit 151, a transform skip flag setting unit 152, an adaptive orthogonal transform identifier setting unit 153, and a context setting unit 154.

[0098] The secondary translation identifier setting unit 151 performs processing related to setting the secondary translation identifier st_idx. For example, the secondary translation identifier setting unit 151 sets, as a coding parameter, a secondary translation identifier for a processing target CU of a frame image held in the sorting buffer 111. In this case, the secondary translation identifier setting unit 151 applies the various methods described in <2-1. Signaling of Secondary Translation Identifier>.

[0099] For example, the secondary translation identifier setting unit 151 sets the secondary translation identifier so that it is stored in the CU header (i.e., so that it is signaled before each TU). Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0100] Furthermore, the secondary transform identifier setting unit 151 can set the secondary transform identifier regardless of the number of non-zero transform coefficients in a CU. In this way, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process. Also, in the decoding process, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0101] Furthermore, the secondary translation identifier setting unit 151 can signal the secondary translation identifier only when the residual PCM coding mode is not applied. This allows the secondary translation identifier to be easily analyzed according to the residual PCM coding mode during decoding, thereby suppressing an increase in the load of the decoding process.

[0102] The secondary transform identifier st_idx set by the secondary transform identifier setting unit 151 as described above is supplied to the orthogonal transform unit 113, the encoding unit 115, and the inverse orthogonal transform unit 118 as Tinfo.

[0103] The transform skip flag setting unit 152 performs processing related to setting of the transform skip flag transform_skip_flag. For example, the transform skip flag setting unit 152 sets, as a coding parameter, a transform skip flag for a processing target TU of a frame image held in the sorting buffer 111. In this case, the transform skip flag setting unit 152 can apply the method described in <2-1. Signaling of Secondary Transform Identifier>.

[0104] For example, the transform skip flag setting unit 152 may set the transform skip flag based on whether or not a secondary transform is performed. That is, st_idx = 0(&& (st_idx[x0][y0] == 0)) may be added to the setting conditions of tsAllowed. In this way, the transform skip flag can be easily set according to the secondary transform identifier. Furthermore, during decoding, the transform skip flag can be easily analyzed according to the secondary transform identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0105] The transform skip flag transform_skip_flag set by the transform skip flag setting unit 152 as described above is supplied to the orthogonal transform unit 113, the encoding unit 115, and the inverse orthogonal transform unit 118 as Tinfo.

[0106] The adaptive orthogonal transform identifier setting unit 153 performs processing related to setting the adaptive orthogonal transform identifier tu_mts_idx. For example, the adaptive orthogonal transform identifier setting unit 153 sets, as a coding parameter, an adaptive orthogonal transform identifier for a processing target TU of a frame image held in the rearrangement buffer 111. In this case, the adaptive orthogonal transform identifier setting unit 153 can apply the technique described in <2-1. Signaling of Secondary Transform Identifier>.

[0107] For example, adaptive orthogonal transform identifier setting unit 153 may set an adaptive orthogonal transform identifier based on whether or not a secondary transform is to be performed. That is, st_idx = 0(&& (st_idx[x0][y0] == 0)) may be added to the setting conditions of mtsAllowed. In this way, an adaptive orthogonal transform identifier can be easily set according to the secondary transform identifier. Furthermore, during decoding, the adaptive orthogonal transform identifier can be easily analyzed according to the secondary transform identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0108] The adaptive orthogonal transform identifier tu_mts_idx set by the adaptive orthogonal transform identifier setting unit 153 as described above is supplied to the orthogonal transform unit 113, the encoding unit 115, and the inverse orthogonal transform unit 118 as Tinfo.

[0109] The context setting unit 154 performs processing related to context setting. For example, the context setting unit 154 derives, as a coding parameter, an initial value (offset) ctxInc of the context used to set a secondary transformation identifier for a processing target CU of a frame image held in the sorting buffer 111. In this case, the context setting unit 154 can apply the technique described in <2-1. Signaling of Secondary Transformation Identifier>.

[0110] For example, the context setting unit 154 may derive the context ctxInc based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index of the first bin of the secondary transform identifier st_idx without requiring complicated processing (more easily).

[0111] The control unit 101 can also generate other encoding parameters. The control unit 101 can also perform any processing other than generating encoding parameters. A description of these will be omitted.

[0112] <Orthogonal transformation section> 19 is a block diagram showing an example of the main configuration of orthogonal transform section 113. In FIG.

[0113] The primary transform unit 171 performs processing related to primary transform. For example, the primary transform unit 171 acquires a prediction residual D supplied from the calculation unit 112. The primary transform unit 171 also performs a predetermined matrix calculation or the like on the prediction residual D to derive transform coefficients Coeff_P (also referred to as primary transform coefficients) after primary transform. That is, the primary transform unit 171 performs primary transform on the prediction residual D.

[0114] The primary transform unit 171 performs this primary transform based on coding parameters such as Tinfo supplied from the control unit 101. That is, the primary transform unit 171 performs the primary transform under the control of the control unit 101. The primary transform unit 171 supplies the primary transform coefficient Coeff_P derived in this manner to the secondary transform unit 172.

[0115] Note that the primary transform unit 171 can also skip (omit) this primary transform under the control of the control unit 101 (in accordance with coding parameters supplied from the control unit 101). In this case, the primary transform unit 171 supplies the prediction residual D to the secondary transform unit 172 as a primary transform coefficient Coeff_P.

[0116] The secondary conversion unit 172 performs processing related to secondary conversion. For example, the secondary conversion unit 172 acquires the primary conversion coefficient Coeff_P supplied from the primary conversion unit 171. The secondary conversion unit 172 performs a predetermined matrix operation or the like on the primary conversion coefficient Coeff_P to derive a conversion coefficient Coeff after the secondary conversion (also referred to as a secondary conversion coefficient). In other words, the secondary conversion unit 172 performs secondary conversion on the primary conversion coefficient Coeff_P.

[0117] The secondary transform unit 172 performs this secondary transform based on coding parameters such as Tinfo supplied from the control unit 101. That is, the secondary transform unit 172 performs the secondary transform under the control of the control unit 101. The secondary transform unit 172 supplies the secondary transform coefficient Coeff derived in this manner (i.e., the transform coefficient Coeff) to the quantization unit 114.

[0118] The secondary transform unit 172 can also skip (omit) this secondary transform under the control of the control unit 101 (according to the coding parameters supplied from the control unit 101). In this case, the secondary transform unit 172 supplies the primary transform coefficient Coeff_P to the quantization unit 114 as the secondary transform coefficient Coeff (i.e., the transform coefficient Coeff).

[0119] The secondary conversion unit 172 includes a rasterization unit 181 , a matrix calculation unit 182 , a scaling unit 183 , a matrix generation unit 184 , and a matrix setting unit 185 .

[0120] The rasterizing unit 181 converts the primary transform coefficients Coeff_P supplied from the primary transforming unit 171 into one-dimensional vectors (1x16 coefficient groups) for each sub-block (4x4 sub-block). The rasterizing unit 181 supplies the obtained one-dimensional vectors to the matrix computing unit 182.

[0121] The matrix calculation unit 182 performs processing related to matrix calculation. For example, the matrix calculation unit 182 acquires a one-dimensional vector supplied from the rasterization unit 181. The matrix calculation unit 182 also acquires a secondary transformation matrix R supplied from the matrix setting unit 185. The matrix calculation unit 182 performs a matrix calculation (matrix multiplication) using the one-dimensional vector and the secondary transformation matrix R. The matrix calculation unit 182 supplies the matrix calculation result (coefficient data obtained by the matrix calculation) to the scaling unit 183.

[0122] The scaling unit 183 performs processing related to scaling of the coefficient data. For example, the scaling unit 183 acquires the coefficient data (matrix operation result) supplied from the matrix operation unit 182. The scaling unit 183 performs clip processing and the like on the coefficient data. The scaling unit 183 supplies the processed coefficient data to the matrix generation unit 184.

[0123] The matrix generation unit 184 performs processing related to matrix generation of one-dimensional vectors. For example, the matrix generation unit 184 obtains scaled 1x16 coefficient data (one-dimensional vectors) supplied from the scaling unit. The matrix generation unit 184 then converts the one-dimensional vectors into 4x4 matrices using a method corresponding to the method used by the rasterization unit 181. The matrix generation unit 184 supplies the matrix (coefficient data group) obtained in this manner to the quantization unit 114 as the conversion coefficients Coeff.

[0124] The matrix setting unit 185 performs processing related to setting of the secondary transformation matrix R used in the matrix calculation unit 182. For example, the matrix setting unit 185 acquires a secondary transformation identifier st_idx set by the control unit 101. Furthermore, the matrix setting unit 185 sets a secondary transformation matrix R based on the secondary transformation identifier st_idx. For example, the matrix setting unit 185 reads out one that corresponds to the secondary transformation identifier st_idx from candidates stored in an internal memory (not shown), and supplies the read out one to the matrix calculation unit 182.

[0125] That is, the secondary conversion unit 172 performs the secondary conversion in accordance with the secondary conversion identifier st_idx set by the control unit 101 (that is, in accordance with the control of the control unit 101).

[0126] <Encoding part> 17 encodes the encoding parameters set by the control unit 101 and generates encoded data (bit stream) including the encoding parameters. In this case, the encoding unit 115 can apply the technique described in <2-1. Signaling of Secondary Translation Identifier>.

[0127] That is, the encoding unit 115 signals the secondary transform identifier st_idx in the CU header (before each TU in the CU). By doing so, as described above, it is possible to suppress an increase in the decoding load.

[0128] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 100 configured as above will be described with reference to the flowchart of FIG.

[0129] When the image encoding process starts, in step S101, the reordering buffer 111 is controlled by the control unit 101 to reorder the frames of the input video data from the display order to the encoding order.

[0130] In step S102, the control unit 101 sets a processing unit for the input image held in the sorting buffer 111 (performs block division).

[0131] In step S103, the control unit 101 determines (sets) coding parameters for the input image held by the reordering buffer 111.

[0132] In step S104, the prediction unit 122 performs a prediction process to generate a predicted image etc. in an optimal prediction mode. For example, in this prediction process, the prediction unit 122 performs intra prediction to generate a predicted image etc. in an optimal intra prediction mode, performs inter prediction to generate a predicted image etc. in an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value etc.

[0133] In step S105, the calculation unit 112 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S104. That is, the calculation unit 112 generates a prediction residual D between the input image and the predicted image. The prediction residual D calculated in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to when the image is encoded as is.

[0134] In step S106, the orthogonal transform unit 113 performs an orthogonal transform process on the prediction residual D generated in the process of step S105, and derives a transform coefficient Coeff.

[0135] In step S107, the quantization unit 114 quantizes the transform coefficient Coeff obtained by the processing in step S106, for example, by using the quantization parameter calculated by the control unit 101, and derives the quantized transform coefficient level level.

[0136] In step S108, the inverse quantization unit 117 inverse quantizes the quantized transform coefficient level generated by the process of step S107 with characteristics corresponding to the quantization characteristics of step S107, to derive the transform coefficient Coeff_IQ.

[0137] In step S109, the inverse orthogonal transform unit 118 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S108 using a method corresponding to the orthogonal transform process of step S106, thereby deriving a prediction residual D'. Note that this inverse orthogonal transform process is similar to the inverse orthogonal transform process (described later) performed on the decoding side, and therefore the explanation (described later) about the decoding side can be applied to the inverse orthogonal transform process of step S109.

[0138] In step S110, the calculation unit 119 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S104 to the prediction residual D' derived in the process of step S109.

[0139] In step S111, the in-loop filter unit 120 performs in-loop filtering on the locally decoded image derived in the processing of step S110.

[0140] In step S112, the frame memory 121 stores the locally decoded image derived by the process of step S110 and the locally decoded image filtered in step S111.

[0141] In step S113, the encoding unit 115 encodes the quantized transform coefficient level LEVEL obtained by the process of step S107. For example, the encoding unit 115 encodes the quantized transform coefficient level LEVEL, which is information related to the image, by arithmetic coding or the like to generate encoded data. At this time, the encoding unit 115 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo). Furthermore, the encoding unit 115 derives residual information RInfo from the quantized transform coefficient level LEVEL and encodes the residual information RInfo.

[0142] In step S114, the accumulation buffer 116 accumulates the encoded data obtained in this manner and outputs it, for example, as a bit stream, to the outside of the image encoding device 100. This bit stream is transmitted to the decoding side, for example, via a transmission path or a recording medium. In addition, the rate control unit 123 performs rate control as necessary.

[0143] When the process of step S114 ends, the image encoding process ends.

[0144] <Encoding parameter setting process flow> An example of the flow of the encoding parameter setting process executed in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 21. The control unit 101 executes this encoding parameter setting process by applying the various methods described in <2-1. Signaling of Secondary Conversion Identifier>.

[0145] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the CU header in step S131. That is, the secondary transformation identifier setting unit 151 sets the secondary transformation identifier so that it is signaled before each TU. Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0146] In this case, the secondary transform identifier setting unit 151 can set the secondary transform identifier regardless of the number of non-zero transform coefficients in the CU. By doing so, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process. Also, in the decoding process, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0147] Furthermore, the secondary translation identifier setting unit 151 can signal the secondary translation identifier only when the residual PCM coding mode is not applied. This allows the secondary translation identifier to be easily analyzed according to the residual PCM coding mode during decoding, thereby suppressing an increase in the load of the decoding process.

[0148] In step S132, the transform skip flag setting unit 152 sets the transform skip flag based on the secondary transform identifier set in step S131. That is, the transform skip flag setting unit 152 sets the transform skip flag based on whether or not to perform secondary transform. In this way, the transform skip flag can be easily set according to the secondary transform identifier. Furthermore, during decoding, the transform skip flag can be easily analyzed according to the secondary transform identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0149] In step S133, adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier based on the secondary transform identifier set in step S131. That is, adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier based on whether or not a secondary transform is to be performed. In this way, it is possible to easily set an adaptive orthogonal transform identifier according to the secondary transform identifier. Furthermore, during decoding, it is possible to easily analyze the adaptive orthogonal transform identifier according to the secondary transform identifier. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0150] In step S134, the context setting unit 154 sets a context without using an adaptive orthogonal transform identifier. That is, the context setting unit 154 derives the context ctxInc based only on the tree type (treeType), without depending on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing.

[0151] When the process of step S134 ends, the encoding parameter setting process ends, and the process returns to FIG.

[0152] In step S132, the transform skip flag setting unit 152 may set the transform skip flag without using a secondary transform identifier. Also, in step S133, the adaptive orthogonal transform identifier setting unit 153 may set an adaptive orthogonal transform identifier without using a secondary transform identifier. Furthermore, in step S134, the context setting unit 154 may set a context based on the adaptive orthogonal transform identifier and the tree type. Also, in this coding parameter setting process, a step of generating other arbitrary coding parameters may be added.

[0153] <Flow of orthogonal transform processing> An example of the flow of the orthogonal transform process executed in step S106 in FIG. 20 will be described with reference to the flowchart in FIG.

[0154] When the orthogonal transform process starts, in step S151, the primary transform unit 171 of the orthogonal transform unit 113 performs primary transform based on the coding parameters set in the process of step S103 in Fig. 20. For example, the primary transform unit 171 performs primary transform using a method specified by the coding parameters. Furthermore, if the coding parameters specify skipping of the primary transform, the primary transform unit 171 skips that primary transform.

[0155] In step S152, the secondary conversion unit 172 performs secondary conversion based on the encoding parameters set in the processing of step S103 in Fig. 20. For example, the secondary conversion unit 172 performs secondary conversion using various parameters set as described with reference to the flowchart in Fig. 21. Note that if skipping of the secondary conversion is specified by a secondary conversion identifier or the like, the secondary conversion unit 172 skips the secondary conversion.

[0156] When the process of step S152 ends, the orthogonal transform process ends, and the process returns to FIG.

[0157] <Encoding process> In step S113 of the image encoding process in Fig. 20, the encoding unit 115 encodes the encoding parameters set in step S103. In this case, the encoding unit 115 can apply the method described in <2-1. Signaling of Secondary Translation Identifier>.

[0158] That is, the encoding unit 115 signals the secondary transform identifier st_idx in the CU header (before each TU in the CU). By doing so, as described above, it is possible to suppress an increase in the decoding load.

[0159] <2-3. Decryption side> <Image decoding device> Next, the decoding side will be described. Fig. 23 is a block diagram showing an example of the configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 200 shown in Fig. 23 is a device that encodes encoded data of a video image. For example, the image decoding device 200 decodes the encoded data using a decoding method described in any of Non-Patent Documents 1 to 10, and generates video data. For example, the image decoding device 200 decodes the encoded data (bit stream) generated by the above-mentioned image encoding device 100, and generates video data.

[0160] Note that Fig. 23 shows the main processing units, data flows, etc., and does not necessarily show everything. That is, in the image decoding device 200, there may be processing units that are not shown as blocks in Fig. 23, and there may be processing or data flows that are not shown as arrows, etc. in Fig. 23.

[0161] 23, the image decoding device 200 includes an accumulation buffer 211, a decoding unit 212, an inverse quantization unit 213, an inverse orthogonal transformation unit 214, a calculation unit 215, an in-loop filter unit 216, a rearrangement buffer 217, a frame memory 218, and a prediction unit 219. The prediction unit 219 includes an intra prediction unit and an inter prediction unit, which are not shown.

[0162] <Accumulation buffer> The accumulation buffer 211 acquires and holds (stores) the bitstream input to the image decoding device 200. The accumulation buffer 211 supplies the accumulated bitstream to the decoding unit 212 at a predetermined timing or when a predetermined condition is met.

[0163] <Decryption section> The decoding unit 212 performs processing related to image decoding. For example, the decoding unit 212 receives a bit stream supplied from the accumulation buffer 211 as input, and performs variable-length decoding of the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, thereby deriving parameters.

[0164] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, etc. That is, the decoding unit 212 parses (analyzes and obtains) this information from the bitstream. These pieces of information will be described below.

[0165] <Header information Hinfo> The header information Hinfo includes header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header (SH), etc. The header information Hinfo includes information that specifies, for example, an image size (horizontal width PicWidth, vertical width PicHeight), a bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum CU size (MaxCUSize) / a minimum CU size (MinCUSize), a maximum depth MaxQTDepth / a minimum depth MinQTDepth of quad-tree partitioning (also referred to as quad-tree partitioning), a maximum depth MaxBTDepth / a minimum depth MinBTDepth of binary-tree partitioning, a maximum transform skip block size MaxTSSize (also referred to as maximum transform skip block size), an on / off flag (also referred to as a valid flag) of each encoding tool, etc.

[0166] For example, the on / off flags for the encoding tools included in the header information Hinfo include on / off flags related to the following conversion and quantization processes. Note that the on / off flags for the encoding tools can also be interpreted as flags indicating whether or not syntax related to the encoding tool is present in the encoded data. Furthermore, if the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable, and if the value of the on / off flag is 0 (false), it indicates that the encoding tool is unusable. Note that the interpretation of the flag values ​​may be reversed.

[0167] Inter-component prediction enabled flag (ccp_enabled_flag): Flag information indicating whether inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available. For example, if this flag information is "1" (true), it indicates that it is available, and if it is "0" (false), it indicates that it is not available.

[0168] This CCP is also called cross-component linear prediction (CCLM or CCLMP).

[0169] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and the like.

[0170] The intra prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from the syntax.

[0171] In addition, the intra-prediction mode information IPinfo includes, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chrominance sample position type identifier (chroma_sample_loc_type_idx), a chrominance MPM identifier (chroma_mpm_idx), and a luminance intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0172] The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether or not inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0173] The multi-class linear prediction mode flag (mclm_flag) is information related to the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single-class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0174] The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier that identifies the type of pixel location of a chroma component (also referred to as a chroma sample location type). For example, if the chroma array type (ChromaArrayType), which is information about the color format, indicates the 420 format, the chroma sample location type identifier is assigned according to the following formula:

[0175] chroma_sample_loc_type_idx == 0:Type2 chroma_sample_loc_type_idx == 1:Type3 chroma_sample_loc_type_idx == 2:Type0 chroma_sample_loc_type_idx == 3:Type1

[0176] Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted (stored in) information about the pixel location of the chrominance component (chroma_sample_loc_info()).

[0177] The chrominance MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chrominance intra prediction mode.

[0178] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0179] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0180] <Conversion information Tinfo> The conversion information Tinfo includes, for example, the following information: Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0181] The width size TBWSize and height TBHSize of the transform block to be processed (or the logarithmic values ​​log2TBWSize and log2TBHSize of TBWSize and TBHSize, respectively, with base 2). Transform skip flag (transform_skip_flag): A flag indicating whether or not to skip the (inverse) primary transform and the (inverse) secondary transform. Scan Identifier (scanIdx) Secondary Transformation Identifier (st_idx) Adaptive orthogonal transform identifier (mts_idx) Quantization parameter (qp) Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax))

[0182] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) includes, for example, the following syntax:

[0183] cbf(coded_block_flag): residual data presence flag last_sig_coeff_x_pos: Last non-zero transform coefficient X coordinate last_sig_coeff_y_pos: Last non-zero transform coefficient Y coordinate coded_sub_block_flag: Sub-block non-zero transform coefficient presence / absence flag sig_coeff_flag: Non-zero transform coefficient presence / absence flag gr1_flag: Flag indicating whether the level of a non-zero transform coefficient is greater than 1 (also called the GR1 flag) gr2_flag: Flag indicating whether the level of a non-zero transform coefficient is greater than 2 (also called the GR2 flag) sign_flag: The sign indicating the sign of a non-zero transform coefficient (also called the sign sign) coeff_abs_level_remaining: Residual level of non-zero transform coefficients (also called non-zero transform coefficient residual level) etc.

[0184] Of course, the residual information Rinfo may include any information, and may include information other than the above information.

[0185] <Filter information Finfo> The filter information Finfo includes, for example, control information related to each of the following filter processes:

[0186] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0187] More specifically, for example, the information includes information specifying the picture to which each filter is applied, an area within the picture, filter On / Off control information for each CU, filter On / Off control information related to slice and tile boundaries, etc. Of course, any information may be included in the filter information Finfo, and information other than these may be included.

[0188] Returning to the description of the decoding unit 212, the decoding unit 212 derives the quantized transform coefficient level "level" at each coefficient position in each transform block by referring to the residual information Rinfo. The decoding unit 212 supplies the quantized transform coefficient level "level" to the inverse quantization unit 213.

[0189] Furthermore, the decoding unit 212 supplies the parsed header information Hinfo, prediction mode information Pinfo, quantized transform coefficient level level, transform information Tinfo, and filter information Finfo to each block.

[0190] The header information Hinfo is supplied to the inverse quantization unit 213 , the inverse orthogonal transformation unit 214 , the prediction unit 219 , and the in-loop filter unit 216 . The prediction mode information Pinfo is supplied to the inverse quantization unit 213 and the prediction unit 219 . The transformation information Tinfo is supplied to the inverse quantization unit 213 and the inverse orthogonal transformation unit 214 . The filter information Finfo is supplied to the in-loop filter unit 216 .

[0191] Of course, the above example is merely an example and is not limiting. For example, each encoding parameter may be supplied to any processing unit. Furthermore, other information may be supplied to any processing unit.

[0192] <Inverse quantization section> The inverse quantization unit 213 has at least a configuration necessary for performing processing related to inverse quantization. For example, the inverse quantization unit 213 receives the transformation information Tinfo and the quantized transformation coefficient level level supplied from the decoding unit 212 as input, scales (inverse quantizes) the value of the quantized transformation coefficient level level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization.

[0193] Note that this inverse quantization is performed as the inverse process of the quantization performed by the quantization unit 114 of the image encoding device 100. Furthermore, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 117 of the image encoding device 100. In other words, the inverse quantization unit 117 of the image encoding device 100 performs the same process (inverse quantization) as the inverse quantization unit 213.

[0194] The inverse quantization unit 213 supplies the derived transform coefficient Coeff_IQ to the inverse orthogonal transformation unit 214 .

[0195] <Inverse orthogonal transform section> The inverse orthogonal transform unit 214 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 214 receives the transform coefficients Coeff_IQ supplied from the inverse quantization unit 213 and the transform information Tinfo supplied from the decoding unit 212 as input, and performs inverse orthogonal transform processing on the transform coefficients Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'.

[0196] This inverse orthogonal transform is performed as the inverse process of the orthogonal transform performed by the orthogonal transform unit 113 of the image encoding device 100. This inverse orthogonal transform is the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 118 of the image encoding device 100. In other words, the inverse orthogonal transform unit 118 of the image encoding device 100 performs the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 214.

[0197] The inverse orthogonal transform unit 214 supplies the derived prediction residual D′ to the calculation unit 215.

[0198] <Arithmetic section> The calculation unit 215 performs processing related to addition of information related to images. For example, the calculation unit 215 receives as input a prediction residual D' supplied from the inverse orthogonal transform unit 214 and a predicted image P supplied from the prediction unit 219. As shown in the following equation, the calculation unit 215 adds the prediction residual D' and the predicted image P (prediction signal) corresponding to the prediction residual D' to derive a locally decoded image Rlocal.

[0199] Rlocal = D' + P

[0200] The calculation unit 215 supplies the derived locally decoded image Rlocal to the in-loop filter unit 216 and the frame memory 218.

[0201] <In-loop filter section> The in-loop filter unit 216 performs processing related to in-loop filter processing. For example, the in-loop filter unit 216 receives as input the locally decoded image Rlocal supplied from the calculation unit 215 and filter information Finfo supplied from the decoding unit 212. Note that any information may be input to the in-loop filter unit 216, and information other than the above information may also be input.

[0202] The in-loop filter unit 216 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo.

[0203] For example, the in-loop filter unit 216 applies four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate.

[0204] The in-loop filter unit 216 performs a filter process corresponding to the filter process performed on the encoding side (for example, the in-loop filter unit 120 of the image encoding device 100). Of course, the filter process performed by the in-loop filter unit 216 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 216 may apply a Wiener filter or the like.

[0205] The in-loop filter unit 216 supplies the filtered locally decoded image Rlocal to a rearrangement buffer 217 and a frame memory 218 .

[0206] <Sorting buffer> The reordering buffer 217 receives the locally decoded image Rlocal supplied from the in-loop filter unit 216 as input and holds (stores) it. The reordering buffer 217 reconstructs a decoded image R for each picture using the locally decoded image Rlocal and holds it (stores it in the buffer). The reordering buffer 217 reorders the obtained decoded images R from the decoding order to the playback order. The reordering buffer 217 outputs the reordered decoded images R to the outside of the image decoding device 200 as video image data.

[0207] <Frame memory> The frame memory 218 performs processing related to the storage of image data. For example, the frame memory 218 receives the locally decoded image Rlocal supplied from the calculation unit 215 as input, reconstructs a decoded image R for each picture, and stores the reconstructed image in a buffer within the frame memory 218.

[0208] Furthermore, the frame memory 218 receives as input the in-loop filtered local decoded image Rlocal supplied from the in-loop filter unit 216, reconstructs a decoded image R for each picture, and stores the reconstructed image in a buffer within the frame memory 218. The frame memory 218 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 219 as a reference image.

[0209] The frame memory 218 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to the generation of decoded images.

[0210] <Prediction Department> The prediction unit 219 performs processing related to generation of a predicted image. For example, the prediction unit 219 receives prediction mode information Pinfo supplied from the decoding unit 212 as input, performs prediction using a prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. When deriving the predicted image P, the prediction unit 219 uses, as a reference image, a decoded image R (or a part thereof) before or after filtering stored in the frame memory 218 and specified by the prediction mode information Pinfo. The prediction unit 219 supplies the derived predicted image P to the calculation unit 215.

[0211] These processing units (accumulation buffer 211 to prediction unit 219) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Also, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0212] The image decoding device 200 decodes the coded data by applying the various techniques described in <2-1. Signaling of Secondary Translation Identifier>. Therefore, as described above, it is possible to suppress an increase in the load of the decoding process.

[0213] <Decryption section> 24 is a block diagram showing an example of the main configuration of the decoding unit 212. As shown in FIG.

[0214] The decoding unit 241 performs processing related to decoding of coded data (bit stream). For example, the decoding unit 241 acquires a bit stream supplied from the accumulation buffer 211. The decoding unit 241 also performs variable-length decoding of the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table. The decoding unit 241 supplies the decoding result to the analysis unit 242.

[0215] The analysis unit 242 performs processing related to parameter analysis. For example, the analysis unit 242 acquires the decoding result supplied from the decoding unit 241. Furthermore, the analysis unit 242 parses (analyzes and acquires) information such as header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, and filter information Finfo from the decoding result.

[0216] Furthermore, the analysis unit 242 derives the quantization transform coefficient level level at each coefficient position in each transform block by referring to the residual information Rinfo, and supplies the quantization transform coefficient level level to the inverse quantization unit 213 .

[0217] The analysis unit 242 includes a secondary transform identifier analysis unit 251 , a transform skip flag analysis unit 252 , an adaptive orthogonal transform identifier analysis unit 253 , and a context setting unit 254 .

[0218] The secondary translation identifier analysis unit 251 performs processing related to analysis of the secondary translation identifier st_idx. For example, the secondary translation identifier analysis unit 251 parses the signaled secondary translation identifier from the decoding result supplied from the decoding unit 241. In this case, the secondary translation identifier analysis unit 251 applies the various methods described in <2-1. Signaling of Secondary Translation Identifier>.

[0219] For example, the secondary translation identifier analysis unit 251 parses the secondary translation identifier stored in the CU header (i.e., signaled before each TU). Therefore, as described above, it is possible to suppress delays in the decoding process and an increase in memory usage, and it is also possible to suppress an increase in the load of the decoding process.

[0220] Furthermore, the secondary transform identifier analyzer 251 can parse the secondary transform identifier regardless of the number of non-zero transform coefficients in the CU. This eliminates the need to count the non-zero transform coefficients, which is a cumbersome process, and thus reduces the load of the decoding process.

[0221] Furthermore, the secondary transform identifier analyzer 251 can parse the secondary transform identifier signaled only when the residual PCM coding mode is not applied. That is, for example, if bdpcm_flag = 1, the secondary transform identifier analyzer 251 determines that the secondary transform is not performed based on this. This makes it easier to analyze the secondary transform identifier, thereby suppressing an increase in the load of the decoding process.

[0222] The secondary transform identifier st_idx parsed by the secondary transform identifier analysis unit 251 as described above is supplied to the inverse quantization unit 213 and the inverse orthogonal transformation unit 214 as Tinfo.

[0223] The transform skip flag analysis unit 252 performs processing related to analysis of the transform skip flag transform_skip_flag. For example, the transform skip flag analysis unit 252 parses the transform skip flag for the TU to be processed as a coding parameter. In this case, the transform skip flag analysis unit 252 can apply the method described in <2-1. Signaling of Secondary Transform Identifier>.

[0224] For example, the transform skip flag analysis unit 252 parses the transform skip flag based on whether or not a secondary transform is to be performed. For example, if a secondary transform is to be performed, the transform skip flag analysis unit 252 determines that a transform skip is not to be performed. In this way, the transform skip flag can be easily analyzed according to the secondary transform identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0225] The transform skip flag transform_skip_flag parsed by the transform skip flag analysis unit 252 as described above is supplied to the inverse quantization unit 213 and the inverse orthogonal transformation unit 214 as Tinfo.

[0226] The adaptive orthogonal transform identifier analysis unit 253 performs processing related to analysis of the adaptive orthogonal transform identifier tu_mts_idx. For example, the adaptive orthogonal transform identifier analysis unit 253 parses the adaptive orthogonal transform identifier for the TU to be processed as a coding parameter. In this case, the adaptive orthogonal transform identifier analysis unit 253 can apply the method described in <2-1. Signaling of Secondary Transform Identifier>.

[0227] For example, the adaptive orthogonal transform identifier analysis unit 253 parses the adaptive orthogonal transform identifier based on whether or not a secondary transform is to be performed. For example, if a secondary transform is to be performed, the adaptive orthogonal transform identifier analysis unit 253 determines that an adaptive orthogonal transform is not to be applied. In this way, the adaptive orthogonal transform identifier can be easily analyzed according to the secondary transform identifier. Therefore, an increase in the load of the decoding process can be suppressed.

[0228] The adaptive orthogonal transform identifier tu_mts_idx set by the adaptive orthogonal transform identifier analyzer 253 as described above is supplied to the inverse quantizer 213 and the inverse orthogonal transformer 214 as Tinfo.

[0229] The context setting unit 254 performs processing related to context setting. For example, the context setting unit 254 derives an initial value (offset) ctxInc of the context used to set a secondary translation identifier for a processing target CU. In this case, the context setting unit 254 can apply the method described in <2-1. Signaling of Secondary Translation Identifier>.

[0230] For example, the context setting unit 254 may derive the context ctxInc based only on the tree type (treeType) without relying on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index of the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0231] The analysis unit 242 can also parse other coding parameters. The analysis unit 242 can also perform any processing other than analyzing coding parameters. A description of these processing will be omitted.

[0232] <Inverse orthogonal transform section> Fig. 25 is a block diagram showing an example of the main configuration of inverse orthogonal transform unit 214 in Fig. 23. As shown in Fig. 25, inverse orthogonal transform unit 214 has an inverse secondary transform unit 271 and an inverse primary transform unit 272.

[0233] The inverse secondary transform unit 271 performs processing related to inverse secondary transform, which is the inverse processing of the secondary transform. For example, the inverse secondary transform unit 271 acquires the transform coefficient Coeff_IQ supplied from the inverse quantization unit 213. Furthermore, the inverse secondary transform unit 271 performs a predetermined matrix operation or the like on the transform coefficient Coeff_IQ to derive the transform coefficient Coeff_IS (also referred to as the primary transform coefficient) after the inverse secondary transform. In other words, the inverse secondary transform unit 271 performs an inverse secondary transform on the transform coefficient Coeff_IQ.

[0234] The inverse secondary transform unit 271 performs this inverse secondary transform based on coding parameters such as Tinfo supplied from the decoding unit 212 (coding parameters signaled in the bitstream parsed by the decoding unit 212). That is, the inverse secondary transform unit 271 performs the inverse secondary transform under the control of the decoding unit 212. The inverse secondary transform unit 271 supplies the primary transform coefficient Coeff_IS derived in this manner to the inverse primary transform unit 272.

[0235] Note that the inverse secondary transform unit 271 can also skip (omit) this inverse secondary transform under the control of the decoding unit 212 (in accordance with the coding parameters supplied from the decoding unit 212). In this case, the inverse secondary transform unit 271 supplies the transform coefficients Coeff_IQ to the inverse primary transform unit 272 as the primary transform coefficients Coeff_IS.

[0236] The inverse primary transform unit 272 performs processing related to inverse primary transform, which is the inverse processing of the primary transform. For example, the inverse primary transform unit 272 obtains the primary transform coefficients Coeff_IS supplied from the inverse secondary transform unit 271. Furthermore, the inverse primary transform unit 272 performs a predetermined matrix operation or the like on the primary transform coefficients Coeff_IS to derive a prediction residual D'.

[0237] The inverse primary transform unit 272 performs this inverse primary transform based on coding parameters such as Tinfo supplied from the decoding unit 212 (coding parameters signaled in the bitstream parsed by the decoding unit 212). That is, the inverse primary transform unit 272 performs the inverse primary transform under the control of the decoding unit 212. The inverse primary transform unit 272 supplies the prediction residual D' derived in this manner to the calculation unit 215.

[0238] Note that the inverse primary transform unit 272 can also skip (omit) this inverse primary transform under the control of the decoding unit 212 (in accordance with the coding parameters supplied from the decoding unit 212). In this case, the inverse primary transform unit 272 supplies the primary transform coefficients Coeff_IS to the calculation unit 215 as prediction residuals D'.

[0239] The inverse secondary transform unit 271 has a rasterization unit 281 , a matrix calculation unit 282 , a scaling unit 283 , a matrix generation unit 284 , and a matrix setting unit 285 .

[0240] The rasterizing unit 281 converts the transform coefficients Coeff_IQ supplied from the inverse quantizing unit 213 into one-dimensional vectors (1x16 coefficient groups) for each sub-block (4x4 sub-block). The rasterizing unit 281 supplies the obtained one-dimensional vectors to the matrix computing unit 282.

[0241] The matrix calculation unit 282 performs processing related to matrix calculations. For example, the matrix calculation unit 282 acquires a one-dimensional vector supplied from the rasterization unit 281. The matrix calculation unit 282 also acquires an inverse secondary transformation matrix IR (=RT) supplied from the matrix setting unit 285. The matrix calculation unit 282 performs a matrix calculation (matrix multiplication) using the one-dimensional vector and the inverse secondary transformation matrix IR. The matrix calculation unit 282 supplies the matrix calculation result (coefficient data obtained by the matrix calculation) to the scaling unit 283.

[0242] The scaling unit 283 performs processing related to scaling of the coefficient data. For example, the scaling unit 283 acquires the coefficient data (matrix operation result) supplied from the matrix operation unit 282. The scaling unit 283 performs clip processing and the like on the coefficient data. The scaling unit 283 supplies the processed coefficient data to the matrix generation unit 284.

[0243] The matrix generation unit 284 performs processing related to matrix generation of a one-dimensional vector. For example, the matrix generation unit 284 obtains scaled 1x16 coefficient data (one-dimensional vector) supplied from the scaling unit 283. The matrix generation unit 284 then converts the one-dimensional vector into a 4x4 matrix using a method corresponding to the method used by the rasterization unit 281. The matrix generation unit 284 supplies the matrix (coefficient data group) obtained in this manner to the inverse primary transformation unit 272 as the primary transformation coefficient Coeff_IS.

[0244] The matrix setting unit 285 performs processing related to setting of the inverse secondary transform matrix IR used in the matrix operation unit 282. For example, the matrix setting unit 285 acquires the secondary transform identifier st_idx supplied from the decoding unit 212. Furthermore, the matrix setting unit 285 sets the inverse secondary transform matrix IR based on the secondary transform identifier st_idx. For example, the matrix setting unit 285 reads out one that corresponds to the secondary transform identifier st_idx (a transposed matrix of the secondary transform matrix R that corresponds to the secondary transform identifier st_idx) from among candidates stored in an internal memory (not shown), and supplies the read out matrix to the matrix operation unit 282.

[0245] That is, the inverse secondary transformation unit 271 performs the inverse secondary transformation in accordance with the secondary transformation identifier st_idx supplied from the decoding unit 212 (that is, in accordance with the control of the decoding unit 212).

[0246] <Flow of image decoding process> Next, a description will be given of the flow of each process executed by the image decoding device 200 configured as above. First, an example of the flow of image decoding process will be described with reference to the flowchart in FIG.

[0247] When the image decoding process starts, in step S201, the accumulation buffer 211 acquires and holds (accumulates) coded data (bitstream) supplied from outside the image decoding device 200.

[0248] In step S202, the decoding unit 212 decodes the coded data (bitstream) to obtain the quantized transform coefficient level level. Furthermore, the decoding unit 212 parses (analyzes and obtains) various coding parameters from the coded data (bitstream) through this decoding.

[0249] In step S203, the inverse quantization unit 213 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S202 to obtain a transform coefficient Coeff_IQ.

[0250] In step S204, the inverse orthogonal transform unit 214 performs inverse orthogonal transform processing, which is the inverse processing of the orthogonal transform processing performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S203, to obtain a prediction residual D'.

[0251] In step S205, the prediction unit 219 performs prediction processing using a prediction method specified by the encoding side based on the information parsed in step S202, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 218.

[0252] In step S206, the calculation unit 215 adds the prediction residual D' obtained in step S204 and the predicted image P obtained in step S205 to derive a locally decoded image Rlocal.

[0253] In step S207, the in-loop filter unit 216 performs in-loop filtering on the locally decoded image Rlocal obtained by the processing in step S206.

[0254] In step S208, the reordering buffer 217 derives decoded images R using the filtered local decoded images R obtained in step S207, and reorders the order of the decoded images R from the decoding order to the playback order. The reordered decoded images R are output as a moving image to the outside of the image decoding device 200.

[0255] In step S209, the frame memory 218 stores at least one of the locally decoded image Rlocal obtained by the process in step S206 and the locally decoded image Rlocal after the filtering process obtained by the process in step S207.

[0256] When the process of step S209 is completed, the image decoding process ends.

[0257] <Decryption process flow> An example of the flow of the decoding process executed in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 27. The decoding unit 212 executes this decoding process by applying the various methods described in <2-1. Signaling of Secondary Translation Identifier>.

[0258] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S231.

[0259] In step S232, the secondary translation identifier analysis unit 251 of the analysis unit 242 analyzes the secondary translation identifier of the CU header (signaled before the TU) from the decoding result.

[0260] By doing so, the secondary transform identifier is obtained before all TUs in the CU are decoded. That is, the inverse quantization and inverse transform processing of each TU can be started before all TUs in the CU are decoded. That is, the buffering period of information required before the inverse quantization and inverse transform processing of the transform block can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. That is, it is possible to suppress an increase in the decoding load.

[0261] In addition, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients in the CU. In this case, the secondary transform identifier analysis unit 251 can analyze the secondary transform identifier regardless of the number of non-zero transform coefficients in the CU. In this way, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0262] Furthermore, the secondary translation identifier may be signaled only when the residual PCM coding mode is not applied. In this case, the secondary translation identifier analysis unit 251 can analyze the secondary translation identifier based on whether the residual PCM coding mode is applied. For example, when the residual PCM coding mode is applied, the secondary translation identifier analysis unit 251 can determine that the secondary translation identifier is not signaled (i.e., the secondary translation is skipped). This makes it easy to analyze the secondary translation identifier, thereby suppressing an increase in the load of the decoding process.

[0263] In step S233, the transform skip flag analysis unit 252 analyzes the transform skip flag. This transform skip flag may be set based on a secondary transform identifier. In this case, the transform skip flag analysis unit 252 can analyze the transform skip flag based on whether or not a secondary transform is to be performed. For example, when a secondary transform is to be performed, the transform skip flag analysis unit 252 can determine that the transform skip flag is not signaled (i.e., transform skip is not to be performed). Therefore, the transform skip flag can be easily analyzed, and an increase in the load of the decoding process can be suppressed.

[0264] In step S234, adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. This adaptive orthogonal transform identifier may be set based on the secondary transform identifier. In this case, adaptive orthogonal transform identifier analysis unit 253 can analyze the adaptive orthogonal transform identifier based on whether or not a secondary transform is to be performed. For example, when a secondary transform is to be performed, adaptive orthogonal transform identifier analysis unit 253 can determine that the adaptive orthogonal transform identifier is not signaled (i.e., adaptive orthogonal transform is not to be performed). Therefore, the adaptive orthogonal transform identifier can be easily analyzed, and an increase in the load of the decoding process can be suppressed.

[0265] In step S235, the context setting unit 254 sets a context without using an adaptive orthogonal transform identifier. That is, the context setting unit 254 derives the context ctxInc based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0266] When the process of step S235 ends, the decoding process ends and the process returns to FIG.

[0267] In step S233, the transform skip flag analysis unit 252 may analyze the transform skip flag without using the secondary transform identifier. Furthermore, in step S234, the adaptive orthogonal transform identifier analysis unit 253 may analyze the adaptive orthogonal transform identifier without using the secondary transform identifier. Furthermore, in step S235, the context setting unit 254 may set a context based on the adaptive orthogonal transform identifier and the tree type. Furthermore, in this decoding process, a step of analyzing any other coding parameters may be added.

[0268] <Flow of inverse orthogonal transform processing> An example of the flow of the inverse orthogonal transform process executed in step S204 in FIG. 26 will be described with reference to the flowchart in FIG.

[0269] When the inverse orthogonal transform process starts, in step S251, the inverse secondary transform unit 271 of the inverse orthogonal transform unit 214 performs an inverse secondary transform based on the coding parameters decoded in the process of step S202 in Fig. 26. For example, the inverse secondary transform unit 271 performs an inverse secondary transform using various coding parameters analyzed as described with reference to the flowchart in Fig. 27. Note that if skipping of the inverse secondary transform is specified by a secondary transform identifier or the like, the inverse secondary transform unit 271 skips that inverse secondary transform.

[0270] In step S252, the inverse primary transform unit 272 performs inverse primary transform based on the coding parameters decoded in the processing of step S202 in Fig. 26. For example, the inverse primary transform unit 272 performs inverse primary transform using a method specified by the coding parameters. Note that if the coding parameters specify skipping of the inverse primary transform, the inverse primary transform unit 272 skips the inverse primary transform.

[0271] When the process of step S252 ends, the inverse orthogonal transform process ends, and the process returns to FIG.

[0272] 3. Second Embodiment <3-1. Signaling of Secondary Transformation Identifier> <TUフッタにおけるシグナリング> The secondary transform identifier st_idx is signaled in the TU footer. That is, the secondary transform identifier is stored in each TU and signaled for each TU. For example, during encoding, the secondary transform identifier is set to be stored in the footer of the TU. In other words, in each TU, the secondary transform identifier is set for each TU to be signaled after the transform coefficients. Also, for example, during decoding, the secondary transform identifier stored in the footer of the TU is analyzed. In other words, the secondary transform identifier for each TU, signaled after the transform coefficients, is analyzed.

[0273] An example of syntax related to the transform unit (transform_unit) in this case is shown in Figure 29. In the example of Figure 29, st_mode (i.e., st_idx) is signaled in the second line from the bottom (the line with gray background). That is, in this case, a secondary transform identifier is set for each TU, and each secondary transform identifier is signaled after the transform coefficient (residual_coding) of the TU to which it corresponds.

[0274] In this way, the secondary transform identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transform identifier corresponding to a certain TU is decoded, the inverse quantization and inverse transform processing of that TU can be started, so that the inverse quantization and inverse transform processing of each TU can be started before all TUs in a CU are decoded.

[0275] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0276] In addition, by signaling a secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TU, which simplifies the counting process of non-zero transform coefficients during encoding and decoding. Therefore, it is possible to suppress an increase in the load of encoding and decoding processes.

[0277] <Signaling control of secondary translation identifier> In this case, since the secondary transform identifier is signaled for each TU, whether or not to signal the secondary transform identifier may be determined based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, whether or not to signal the secondary transform identifier can be controlled for each TU.

[0278] An example of syntax related to st_mode in this case is shown in Figure 30. In this case, as shown in the second row from the top (gray row), for stAllowed to be true, the smaller of the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight) must be equal to or greater than a predetermined threshold (minStSize). In other words, for transform blocks with transform block sizes smaller than the threshold, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the coding block size, the secondary transform identifier may be set based on the transform block size.

[0279] <Separation of luminance and color difference> A TU includes a transform block (TB) of luma (Y) and a transform block of chroma (Cb, Cr). Therefore, the secondary transform identifier for luma and the secondary transform identifier for chroma may be signaled separately. That is, the secondary transform identifier corresponding to the luma TB and the secondary transform identifier corresponding to the chroma TB included in each TU may be signaled in the footer of the TU.

[0280] For example, during encoding, a secondary transform identifier corresponding to the luma TB and a secondary transform identifier corresponding to the chroma TB included in each TU are set to be stored in the footer of the TU (signaled after the transform coefficients).Furthermore, for example, during decoding, the secondary transform identifier corresponding to the luma TB and the secondary transform identifier corresponding to the chroma TB included in the TU, which are stored in the footer of the TU (signaled after the transform coefficients), are analyzed.

[0281] In this case, too, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0282] <3-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 18. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0283] <Encoding parameter setting process flow> Furthermore, the image coding device 100 in this case performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Furthermore, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0284] An example of the flow of the encoding parameter setting process executed by the control unit 101 in this case in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 31. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <3-1. Signaling of Secondary Conversion Identifier>.

[0285] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TU footer in step S301. That is, the secondary transformation identifier setting unit 151 sets the secondary transformation identifier for each TU so that it is signaled after the transform coefficients of that TU. Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0286] In addition, by signaling a secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TU, which simplifies the counting process of non-zero transform coefficients during encoding and decoding. Therefore, it is possible to suppress an increase in the load of the encoding process.

[0287] Furthermore, the secondary transform identifier setting unit 151 may determine whether to signal the secondary transform identifier based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, it is possible to control whether to signal the secondary transform identifier for each TU.

[0288] Furthermore, the secondary transform identifier setting unit 151 may set a secondary transform identifier corresponding to the luminance TB and a secondary transform identifier corresponding to the chrominance TB included in each TU so that they are stored in the footer of the TU (so that they are signaled after the transform coefficients).

[0289] In step S302, the transform skip flag setting unit 152 sets a transform skip flag. In step S303, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier. In step S304, the context setting unit 154 sets a context. When the processing of step S304 ends, the encoding parameter setting processing ends, and the processing returns to FIG. 20.

[0290] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0291] <3-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 24. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0292] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0293] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 32. In this case, the decoding unit 212 executes this decoding process by applying the various methods described in <3-1. Signaling of Secondary Translation Identifier>.

[0294] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S321.

[0295] In step S322, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier in the TU footer (signaled for each TU after the transform coefficients of the TU) from the decoding result.

[0296] In this way, the secondary transform identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transform identifier corresponding to a certain TU is decoded, the inverse quantization and inverse transform processing of that TU can be started, so that the inverse quantization and inverse transform processing of each TU can be started before all TUs in a CU are decoded.

[0297] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0298] In addition, by signaling the secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count the non-zero transform coefficients within the TU, so that the counting process of non-zero transform coefficients during decoding can be simplified. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0299] Furthermore, the secondary transform identifier analysis unit 251 may determine whether or not a secondary transform identifier has been signaled based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, it is possible to determine whether or not a secondary transform identifier has been signaled for each TU.

[0300] Furthermore, the secondary transform identifier may be signaled separately as a secondary transform identifier for luma and a secondary transform identifier for chroma. In this case, the secondary transform identifier analysis unit 251 analyzes the secondary transform identifier for luma and the secondary transform identifier for chroma in the TU footer (signaled for each TU after the transform coefficients of the TU).

[0301] In step S323, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S324, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. In step S325, the context setting unit 254 sets a context. When the processing of step S325 ends, the decoding processing ends, and the processing returns to FIG. 26.

[0302] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0303] 4. Third Embodiment <4-1. Secondary Transformation Identifier Signaling> <TUヘッダにおけるシグナリング> The secondary transform identifier st_idx is signaled in the TU header. That is, the secondary transform identifier is stored in each TU and signaled for each TU. For example, during encoding, the secondary transform identifier is set to be stored in the header of the TU. In other words, in each TU, the secondary transform identifier is set to be signaled before the transform coefficients. Also, for example, during decoding, the secondary transform identifier stored in the header of the TU is analyzed. In other words, the secondary transform identifier for each TU, signaled before the transform coefficients in each TU, is analyzed.

[0304] An example of syntax related to the transform unit (transform_unit) in this case is shown in Figure 33. In the example of Figure 33, st_mode (i.e., st_idx) is signaled in the 15th and 16th lines from the top (gray lines). That is, in this case, a secondary transform identifier is set for each TU, and each secondary transform identifier is signaled before the transform coefficient (residual_coding) of the TU to which it corresponds.

[0305] In this way, the secondary transform identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transform identifier corresponding to a certain TU is decoded, the inverse quantization and inverse transform processing of that TU can be started, so that the inverse quantization and inverse transform processing of each TU can be started before all TUs in a CU are decoded.

[0306] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0307] In addition, by signaling a secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TU, which simplifies the counting process of non-zero transform coefficients during encoding and decoding. Therefore, it is possible to suppress an increase in the load of encoding and decoding processes.

[0308] <Signaling control of secondary translation identifier> Also, as in the second embodiment, in this case, since the secondary transform identifier is signaled for each TU, whether or not to signal the secondary transform identifier may be determined based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, whether or not to signal the secondary transform identifier can be controlled for each TU.

[0309] An example of syntax related to st_mode in this case is shown in Figure 34. In this case, as shown in the second row from the top (gray row), for stAllowed to be true, the smaller of the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight) must be equal to or greater than a predetermined threshold (minStSize). In other words, for transform blocks with transform block sizes smaller than the threshold, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the coding block size, the secondary transform identifier may be set based on the transform block size.

[0310] <Omission of counting non-zero transform coefficients> Furthermore, as in the first embodiment, in this case, the conditional expression that refers to the number of non-zero transform coefficients may be deleted from the decoding / encoding condition of the secondary transform identifier. For example, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients. Also, for example, the secondary transform identifier may be analyzed regardless of the number of non-zero transform coefficients.

[0311] In this way, when deriving a secondary transform identifier in the encoding process, the cumbersome counting of non-zero transform coefficients can be omitted, thereby suppressing an increase in the load of the encoding process.Furthermore, when analyzing a secondary transform identifier in the decoding process, the cumbersome counting of non-zero transform coefficients can be omitted, thereby suppressing an increase in the load of the decoding process.

[0312] <Setting the secondary transform identifier based on the residual PCM mode> Also, as in the first embodiment, in this case, the secondary transform identifier may be signaled only when the residual PCM coding mode is not applied (i.e., when the residual PCM coding mode is applied, the secondary transform identifier is not signaled).

[0313] This allows the secondary conversion identifier to be easily set according to the residual PCM coding mode. Furthermore, during decoding, the secondary conversion identifier can be easily analyzed according to the residual PCM coding mode. This reduces the load of the decoding process.

[0314] In the example of Figure 34, as shown in the second row from the top (gray row), bdpcm_flag = 0 is required for stAllowed to be true. In other words, when the residual PCM coding mode is applied, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients, the secondary transform identifier may be set based on a flag indicating whether the residual PCM coding mode is applied.

[0315] Of course, the condition when this bdpcm_flag is not applied (bdpcm_flag == 0) may be excluded from the determination of whether or not to perform secondary conversion (deriving stAllowed).

[0316] <Context derivation> Also, as in the first embodiment, in this case, the initial value (offset) of the context ctxInc may be derived based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx (FIG. 16).

[0317] By doing so, it is possible to derive the context index of the first bin of the secondary transformation identifier st_idx without requiring complicated processing (more easily). Similarly, it is possible to derive the context index of the first bin of the secondary transformation identifier st_idx more easily during decoding. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0318] <Separation of luminance and color difference> A TU includes a transform block (TB) of luma (Y) and a transform block of chroma (Cb, Cr). Therefore, the secondary transform identifier for luma and the secondary transform identifier for chroma may be signaled separately. That is, the secondary transform identifier corresponding to the luma TB and the secondary transform identifier corresponding to the chroma TB included in each TU may be signaled in the header of the TU.

[0319] For example, during encoding, a secondary transform identifier corresponding to the luma TB and a secondary transform identifier corresponding to the chroma TB included in each TU are set to be stored in the header of the TU (signaled before the transform coefficients).Furthermore, for example, during decoding, the secondary transform identifier corresponding to the luma TB and the secondary transform identifier corresponding to the chroma TB included in the TU, which are stored in the header of the TU (signaled before the transform coefficients), are analyzed.

[0320] In this case, too, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0321] <4-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 18. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0322] <Encoding parameter setting process flow> Furthermore, the image coding device 100 in this case performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Furthermore, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0323] An example of the flow of the encoding parameter setting process executed by the control unit 101 in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 35. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <4-1. Signaling of Secondary Conversion Identifier>.

[0324] When the coding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TU header in step S341. That is, the secondary transformation identifier setting unit 151 sets the secondary transformation identifier for each TU so that it is signaled before the transformation coefficients of that TU. Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0325] In addition, by signaling a secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TU, which simplifies the counting process of non-zero transform coefficients during encoding and decoding. Therefore, it is possible to suppress an increase in the load of the encoding process.

[0326] Furthermore, the secondary transform identifier setting unit 151 may determine whether to signal the secondary transform identifier based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, it is possible to control whether to signal the secondary transform identifier for each TU.

[0327] Furthermore, the secondary transform identifier setting unit 151 may set the secondary transform identifier regardless of the number of non-zero transform coefficients. By doing so, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process. Also, in the decoding process, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0328] Alternatively, the secondary translation identifier setting unit 151 may signal the secondary translation identifier only when the residual PCM coding mode is not applied. This allows the secondary translation identifier to be easily analyzed according to the residual PCM coding mode during decoding, thereby suppressing an increase in the load of the decoding process.

[0329] Furthermore, the secondary transform identifier setting unit 151 may set a secondary transform identifier corresponding to the luminance TB and a secondary transform identifier corresponding to the chrominance TB included in each TU so that they are stored in the header of the TU (so that they are signaled before the transform coefficients).

[0330] In step S342, the transform skip flag setting unit 152 sets a transform skip flag, and in step S343, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier.

[0331] In step S344, the context setting unit 154 sets a context without using an adaptive orthogonal transform identifier. That is, the context setting unit 154 derives the context ctxInc based only on the tree type (treeType), without depending on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing.

[0332] When the process of step S344 ends, the encoding parameter setting process ends, and the process returns to FIG.

[0333] In step S344, the context setting unit 154 may set a context based on the adaptive orthogonal transform identifier and the tree type. Also, in this coding parameter setting process, a step of generating other arbitrary coding parameters may be added.

[0334] <4-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 24. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0335] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0336] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 36. In this case, the decoding unit 212 executes this decoding process by applying the various methods described in <4-1. Signaling of Secondary Translation Identifier>.

[0337] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S361.

[0338] In step S362, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier of the TU header (signaled for each TU before the transform coefficients of the TU) from the decoding result.

[0339] In this way, the secondary transform identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transform identifier corresponding to a certain TU is decoded, the inverse quantization and inverse transform processing of that TU can be started, so that the inverse quantization and inverse transform processing of each TU can be started before all TUs in a CU are decoded.

[0340] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TU can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0341] In addition, by signaling the secondary transform identifier for each TU, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count the non-zero transform coefficients within the TU, so that the counting process of non-zero transform coefficients during decoding can be simplified. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0342] Furthermore, the secondary transform identifier analysis unit 251 may determine whether or not a secondary transform identifier has been signaled based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, it is possible to determine whether or not a secondary transform identifier has been signaled for each TU.

[0343] Furthermore, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients. In this case, the secondary transform identifier analysis unit 251 can analyze the secondary transform identifier regardless of the number of non-zero transform coefficients. In this way, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0344] Alternatively, the secondary translation identifier may be signaled only when the residual PCM coding mode is not applied. In this case, the secondary translation identifier analyzer 251 can analyze the secondary translation identifier based on whether the residual PCM coding mode is applied. For example, when the residual PCM coding mode is applied, the secondary translation identifier analyzer 251 can determine that the secondary translation identifier is not signaled (i.e., the secondary translation is skipped). This allows the secondary translation identifier to be easily analyzed, thereby suppressing an increase in the load of the decoding process.

[0345] Furthermore, the secondary transform identifier may be signaled separately as a secondary transform identifier for luma and a secondary transform identifier for chroma. In this case, the secondary transform identifier analysis unit 251 analyzes the secondary transform identifier for luma and the secondary transform identifier for chroma in the TU header (signaled for each TU before the transform coefficients of that TU).

[0346] In step S363, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S364, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier.

[0347] In step S365, the context setting unit 254 sets a context without using an adaptive orthogonal transform identifier. That is, the context setting unit 254 derives the context ctxInc based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0348] When the process of step S365 ends, the decoding process ends and the process returns to FIG.

[0349] In step S365, the context setting unit 254 may set the context based on the adaptive orthogonal transform identifier and the tree type. Also, in this decoding process, a step of analyzing any other coding parameters may be added.

[0350] 5. Fourth Embodiment <5-1. Secondary Transformation Identifier Signaling> <TBフッタにおけるシグナリング> The secondary transform identifier st_idx is signaled in the TB footer. That is, the secondary transform identifier is stored in each TB and signaled for each TB. That is, the secondary transform identifier corresponding to each TB is signaled for each of the luminance (Y) TB, chrominance (Cr) TB, and chrominance (Cb) TB.

[0351] For example, during encoding, a secondary transform identifier is set so as to be stored in the footer of the TB. In other words, a secondary transform identifier for each TB is set so as to be signaled after the transform coefficients in each TB. Furthermore, for example, during decoding, the secondary transform identifier stored in the footer of the TB is analyzed. In other words, the secondary transform identifier for each TB, which is signaled after the transform coefficients in each TB, is analyzed.

[0352] An example of syntax related to the transform block (residual_coding) in this case is shown in Figure 37. In the example of Figure 37, st_mode (i.e., st_idx) is signaled in the second line from the bottom (gray line). That is, in this case, a secondary transform identifier is set for each TB (for each cIdx that identifies a component). Furthermore, the secondary transform identifier is signaled after the transform coefficient of the TB to which it corresponds.

[0353] In this way, the secondary transformation identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transformation identifier corresponding to a certain TB is decoded, the inverse quantization and inverse transformation process of that TB can be started, so that the inverse quantization and inverse transformation process of each TB can be started before all TUs in a CU are decoded.

[0354] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of the transform block can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0355] In addition, by signaling a secondary transform identifier for each TB, the counting of non-zero transform coefficients is closed to the TB to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, so that the counting process of non-zero transform coefficients during encoding and decoding can be simplified. Therefore, it is possible to suppress an increase in the load of encoding and decoding processes.

[0356] <Signaling control of secondary translation identifier> In this case, since the secondary transformation identifier is signaled for each TB, whether or not to signal the secondary transformation identifier may be determined based on the transformation block size (the horizontal transformation block size (tbWidth) and the vertical transformation block size (tbHeight)). In this way, whether or not to signal the secondary transformation identifier can be controlled for each TB.

[0357] An example of syntax related to st_mode in this case is shown in Figure 38. In this case, as shown in the second row from the top (gray row), for stAllowed to be true, the smaller of the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight) must be equal to or greater than a predetermined threshold (minStSize). In other words, for transform blocks with transform block sizes smaller than the threshold, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the coding block size, the secondary transform identifier may be set based on the transform block size.

[0358] <Context derivation> In this case, the initial value (offset) of the context ctxInc may be derived based on the component (for example, based on the value of the identifier cIdx that identifies the component).

[0359] For example, the initial context value (offset) ctxInc may be derived based on whether the component is luma (Y) or chroma (Cb or Cr). Because the probability of applying a secondary transform differs between luma and chroma, doing so enables more efficient encoding and decoding of the secondary transform identifier st_idx for each component.

[0360] Alternatively, the initial context value (offset) ctxInc may be derived based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr). In this way, the context variable ctx can be set in more detail, and the secondary transform identifier st_idx can be coded and decoded more efficiently.

[0361] For example, in the case of A in Fig. 39, the initial value (offset) of the context ctxInc is set to 0 or 1 based on whether the value of the identifier cIdx that identifies the component is luminance (Y) or chrominance (Cb or Cr). Also, for example, in the case of B in Fig. 39, the value of the identifier cIdx that identifies the component is set to the initial value (offset) of the context ctxInc.

[0362] <5-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 18. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0363] <Encoding parameter setting process flow 1> Furthermore, the image coding device 100 in this case performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Furthermore, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0364] An example of the flow of the encoding parameter setting process executed by the control unit 101 in this case in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 40. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <5-1. Signaling of Secondary Conversion Identifier>. In this example, the initial value (offset) ctxInc of the context is derived based on whether the component is luminance (Y) or chrominance (Cb or Cr) (A in Fig. 39).

[0365] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TB footer in step S401. That is, the secondary transformation identifier setting unit 151 sets the secondary transformation identifier for each TB so that it is signaled after the transform coefficients of that TB. Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0366] In addition, by signaling a secondary transform identifier for each TB, the counting of non-zero transform coefficients is closed to the TB to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, so that the counting process of non-zero transform coefficients during encoding and decoding can be simplified. Therefore, it is possible to suppress an increase in the load of the encoding process.

[0367] Furthermore, the secondary transformation identifier setting unit 151 may determine whether to signal the secondary transformation identifier based on the transformation block size (the horizontal transformation block size (tbWidth) and the vertical transformation block size (tbHeight)). In this way, it is possible to control whether to signal the secondary transformation identifier for each TB.

[0368] In step S402, the transform skip flag setting unit 152 sets a transform skip flag, and in step S403, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier.

[0369] In step S404, the context setting unit 154 references the identifier cIdx that identifies the component, and sets a context based on whether the component is luminance (Y) or chrominance (Cb or Cr). When the process of step S404 ends, the encoding parameter setting process ends, and the process returns to FIG. 20.

[0370] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0371] <Encoding parameter setting process flow 2> Another example of the flow of the encoding parameter setting process executed by the control unit 101 in this case will be described with reference to the flowchart in Fig. 41. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <5-1. Signaling of Secondary Transform Identifier>. In this example, the initial value (offset) ctxInc of the context is derived based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr) (B in Fig. 39).

[0372] In this case, the processes of steps S421 to S423 are executed in the same manner as the processes of steps S401 to S403 in FIG.

[0373] In step S424, the context setting unit 154 references the identifier cIdx that identifies the component, and sets a context based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr). When the process of step S424 ends, the encoding parameter setting process ends, and the process returns to FIG. 20.

[0374] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0375] <5-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 24. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0376] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0377] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 42. In this case, the decoding unit 212 executes this decoding process by applying the various methods described in <5-1. Signaling of Secondary Translation Identifier>.

[0378] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S441.

[0379] In step S442, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier of the TB footer (signaled for each TB after the transform coefficients of that TB) from the decoding result.

[0380] In this way, the secondary transformation identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transformation identifier corresponding to a certain TB is decoded, the inverse quantization and inverse transformation process of that TB can be started, so that the inverse quantization and inverse transformation process of each TB can be started before all TUs in a CU are decoded.

[0381] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TB can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0382] In addition, by signaling the secondary transform identifier for each TB, the counting of non-zero transform coefficients is limited to the TB to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, so that the counting process of non-zero transform coefficients during decoding can be simplified. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0383] Furthermore, the secondary transform identifier analysis unit 251 may determine whether or not a secondary transform identifier has been signaled based on the transform block size (the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight)). In this way, it is possible to determine whether or not a secondary transform identifier has been signaled for each TU.

[0384] In step S443, the transform skip flag analysis unit 252 analyzes the transform skip flag.In step S444, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier.

[0385] In step S445, the context setting unit 254 sets a context. At this time, similar to the processing of step S404 (FIG. 40), the context setting unit 254 may set the context based on whether the component is luminance (Y) or chrominance (Cb or Cr) by referring to the identifier cIdx that identifies the component. Furthermore, similar to the processing of step S424 (FIG. 41), the context setting unit 254 may set the context based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr) by referring to the identifier cIdx that identifies the component.

[0386] By setting it in this way, the secondary translation identifier st_idx can be decoded more efficiently.

[0387] When the process of step S445 ends, the decoding process ends and the process returns to FIG.

[0388] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0389] 6. Fifth Embodiment <6-1. Secondary Transformation Identifier Signaling> <TBヘッダにおけるシグナリング> The secondary transform identifier st_idx is signaled in the TB header. That is, the secondary transform identifier is stored in each TB and signaled for each TB. For example, during encoding, the secondary transform identifier is set so as to be stored in the TB header. In other words, in each TB, the secondary transform identifier for each TB is set so as to be signaled before the transform coefficients. Also, for example, during decoding, the secondary transform identifier stored in the TB header is analyzed. In other words, the secondary transform identifier for each TB, signaled before the transform coefficients in each TB, is analyzed.

[0390] An example of syntax related to the transform block (residual_coding) in this case is shown in Figure 43. In the example of Figure 43, st_mode (i.e., st_idx) is signaled in the second line from the top (gray line). That is, in this case, a secondary transform identifier is set for each TB (for each cIdx that identifies a component). Furthermore, the secondary transform identifier is signaled before the transform coefficient of the TB to which it corresponds.

[0391] In this way, the secondary transformation identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transformation identifier corresponding to a certain TB is decoded, the inverse quantization and inverse transformation process of that TB can be started, so that the inverse quantization and inverse transformation process of each TB can be started before all TUs in a CU are decoded.

[0392] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of the transform block can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0393] In addition, by signaling a secondary transform identifier for each TB, the counting of non-zero transform coefficients is closed to the TB to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, so that the counting process of non-zero transform coefficients during encoding and decoding can be simplified. Therefore, it is possible to suppress an increase in the load of encoding and decoding processes.

[0394] <Signaling control of secondary translation identifier> In this case, since the secondary transformation identifier is signaled for each TB, whether or not to signal the secondary transformation identifier may be determined based on the transformation block size (the horizontal transformation block size (tbWidth) and the vertical transformation block size (tbHeight)). In this way, whether or not to signal the secondary transformation identifier can be controlled for each TB.

[0395] An example of syntax related to st_mode in this case is shown in Figure 44. In this case, as shown in the second row from the top (gray row), for stAllowed to be true, the smaller of the horizontal transform block size (tbWidth) and the vertical transform block size (tbHeight) must be equal to or greater than a predetermined threshold (minStSize). In other words, for transform blocks with transform block sizes smaller than the threshold, the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the coding block size, the secondary transform identifier may be set based on the transform block size.

[0396] <Omission of counting non-zero transform coefficients> Furthermore, as in the first embodiment, in this case, the conditional expression that refers to the number of non-zero transform coefficients may be deleted from the decoding / encoding condition of the secondary transform identifier (FIG. 44). For example, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients. Also, for example, the secondary transform identifier may be analyzed regardless of the number of non-zero transform coefficients.

[0397] In this way, when deriving a secondary transformation identifier in the encoding process, the cumbersome counting of non-zero coefficients can be omitted, thereby suppressing an increase in the load of the encoding process.Furthermore, when analyzing a secondary transformation identifier in the decoding process, the cumbersome counting of non-zero coefficients can be omitted, thereby suppressing an increase in the load of the decoding process.

[0398] Note that the counting of non-zero transform coefficients can also be omitted in the TB syntax (residual_coding) in FIG. 43 (sixth line from the bottom (gray line) in FIG. 43).

[0399] <Setting the secondary conversion identifier based on the conversion skip flag> Also, in this case, the secondary translation identifier may be signaled only when the translation skip is not applied (ie, when the translation skip is applied, the secondary translation identifier is not signaled).

[0400] In this way, the secondary transformation identifier can be easily set according to the transformation skip flag. Also, during decoding, the secondary transformation identifier can be easily analyzed according to the transformation skip flag. Therefore, an increase in the load of the decoding process can be suppressed.

[0401] For example, in the second line from the top of FIG. 44, the condition for stAllowed to be true includes that transform skip is not applied (&& ! transform_skip_flag[x0][y0][cIdx]). In other words, for stAllowed to be true, transform skip must not be applied. In this way, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients, the secondary transform identifier may be set based on a flag indicating whether to apply transform skip.

[0402] <Context derivation> Also, as in the first embodiment, in this case, the initial value (offset) of the context ctxInc may be derived based only on the tree type (treeType) without depending on the adaptive orthogonal transform identifier tu_mts_idx (FIG. 16).

[0403] By doing so, it is possible to derive the context index of the first bin of the secondary transformation identifier st_idx without requiring complicated processing (more easily). Similarly, it is possible to derive the context index of the first bin of the secondary transformation identifier st_idx more easily during decoding. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0404] Also, in this case, similarly to the fourth embodiment, the initial value (offset) ctxInc of the context may be derived based on the component (for example, based on the value of the identifier cIdx that identifies the component) (FIG. 39).

[0405] For example, the initial context value (offset) ctxInc may be derived based on whether the component is luma (Y) or chroma (Cb or Cr). Because the probability of applying a secondary transform differs between luma and chroma, doing so enables more efficient encoding and decoding of the secondary transform identifier st_idx for each component.

[0406] Alternatively, the initial context value (offset) ctxInc may be derived based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr). In this way, the context variable ctx can be set in more detail, and the secondary transform identifier st_idx can be coded and decoded more efficiently.

[0407] <6-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 18. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0408] <Encoding parameter setting process flow> Furthermore, the image coding device 100 in this case performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Furthermore, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0409] An example of the flow of the encoding parameter setting process executed by the control unit 101 in this case in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 45. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <6-1. Signaling of Secondary Conversion Identifier>.

[0410] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TB header in step S461. That is, the secondary transformation identifier setting unit 151 sets the secondary transformation identifier for each TB so that it is signaled before the transformation coefficients of that TB. Therefore, as described above, it is possible to suppress delays in the decoding process and increases in memory usage, and it is also possible to suppress increases in the decoding process load.

[0411] In addition, by signaling a secondary transform identifier for each TB, the counting of non-zero transform coefficients is limited to the TU to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, which simplifies the counting process of non-zero transform coefficients during encoding and decoding. Therefore, it is possible to suppress an increase in the load of the encoding process.

[0412] Furthermore, the secondary transformation identifier setting unit 151 may determine whether to signal the secondary transformation identifier based on the transformation block size (the horizontal transformation block size (tbWidth) and the vertical transformation block size (tbHeight)). In this way, it is possible to control whether to signal the secondary transformation identifier for each TB.

[0413] Furthermore, the secondary transform identifier setting unit 151 may set the secondary transform identifier regardless of the number of non-zero transform coefficients. By doing so, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process. Also, in the decoding process, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0414] Furthermore, the secondary transformation identifier setting unit 151 may signal the secondary transformation identifier only when the transformation skip is not applied. By doing so, the secondary transformation identifier can be easily analyzed according to the transformation skip flag during decoding, and an increase in the load of the decoding process can be suppressed.

[0415] In step S462, the transform skip flag setting unit 152 sets a transform skip flag, and in step S463, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier.

[0416] In step S464, the context setting unit 154 sets a context. At this time, for example, the context setting unit 154 may set a context without using an adaptive orthogonal transform identifier. That is, in this case, the context setting unit 154 derives the context ctxInc based only on the tree type (treeType) without relying on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing.

[0417] The context setting unit 154 may set a context based on whether the component is luminance (Y) or chrominance (Cb or Cr) by referring to the identifier cIdx that identifies the component. Furthermore, the context setting unit 154 may set a context based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr) by referring to the identifier cIdx that identifies the component. In this way, the secondary transform identifier st_idx for each component can be coded more efficiently.

[0418] When the process of step S464 ends, the encoding parameter setting process ends, and the process returns to FIG.

[0419] In step S464, the context setting unit 154 may set a context based on the adaptive orthogonal transform identifier and the tree type. In addition, a step of generating other arbitrary coding parameters may be added to this coding parameter setting process.

[0420] <6-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 24. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0421] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0422] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 46. In this case, the decoding unit 212 executes this decoding process by applying the various methods described in <6-1. Signaling of Secondary Translation Identifier>.

[0423] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S481.

[0424] In step S482, the secondary transform identifier analyzer 251 of the analyzer 242 analyzes the secondary transform identifier of the TB header (signaled for each TB before the transform coefficients of that TB) from the decoding result.

[0425] In this way, the secondary transformation identifier can be decoded before all TUs in a CU are decoded. In other words, once the secondary transformation identifier corresponding to a certain TB is decoded, the inverse quantization and inverse transformation process of that TB can be started, so that the inverse quantization and inverse transformation process of each TB can be started before all TUs in a CU are decoded.

[0426] Therefore, the buffering period of information required before starting the inverse quantization and inverse transform processing of each TB can be made shorter than when signaling is performed in the footer of the CU. Therefore, it is possible to suppress an increase in delay and memory usage. In other words, it is possible to suppress an increase in the decoding load.

[0427] In addition, by signaling the secondary transform identifier for each TB, the counting of non-zero transform coefficients is limited to the TB to be processed. In other words, it is only necessary to count non-zero transform coefficients within the TB, so that the counting process of non-zero transform coefficients during decoding can be simplified. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0428] Furthermore, the secondary transformation identifier analysis unit 251 may determine whether or not a secondary transformation identifier has been signaled based on the transformation block size (the horizontal transformation block size (tbWidth) and the vertical transformation block size (tbHeight)). In this way, it is possible to determine whether or not a secondary transformation identifier has been signaled for each TB.

[0429] Furthermore, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients. In this case, the secondary transform identifier analysis unit 251 can analyze the secondary transform identifier regardless of the number of non-zero transform coefficients. In this way, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0430] Furthermore, the secondary translation identifier may be signaled only when the translation skip is not applied. In this case, the secondary translation identifier analysis unit 251 can analyze the secondary translation identifier based on whether the translation skip is applied or not. For example, when the translation skip is applied, the secondary translation identifier analysis unit 251 can determine that the secondary translation identifier is not signaled (i.e., the secondary translation is skipped). Therefore, the secondary translation identifier can be easily analyzed, and an increase in the load of the decoding process can be suppressed.

[0431] In step S483, the transform skip flag analysis unit 252 analyzes the transform skip flag.In step S484, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier.

[0432] In step S485, the context setting unit 254 sets a context. At this time, the context setting unit 254 may set a context without using an adaptive orthogonal transform identifier. That is, the context setting unit 254 may derive the context ctxInc based only on the tree type (treeType) without relying on the adaptive orthogonal transform identifier tu_mts_idx. In this way, it is possible to derive the context index for the first bin of the secondary transform identifier st_idx without requiring (more easily deriving) complicated processing. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0433] The context setting unit 254 may set a context based on whether the component is luminance (Y) or chrominance (Cb or Cr) by referring to the identifier cIdx that identifies the component. Furthermore, the context setting unit 254 may set a context based on whether the component is luminance (Y), chrominance (Cb), or chrominance (Cr) by referring to the identifier cIdx that identifies the component. In this way, the secondary transform identifier st_idx for each component can be coded more efficiently.

[0434] When the process of step S485 ends, the decoding process ends and the process returns to FIG.

[0435] In step S485, the context setting unit 254 may set the context based on the adaptive orthogonal transform identifier and the tree type. Also, in this decoding process, a step of analyzing any other coding parameters may be added.

[0436] 7. Sixth Embodiment <7-1. Secondary Transformation Identifier Signaling> <Block size limit> The secondary transformation identifier is signaled only when the information about the block size is equal to or less than a threshold. In other words, the secondary transformation identifier is set so that the secondary transformation is performed only when the information about the block size is equal to or less than a predetermined threshold. In other words, the secondary transformation is performed only on blocks equal to or less than a predetermined size.

[0437] By doing so, it is possible to skip (omit) the secondary transform for coding blocks with a block size larger than a predetermined size. That is, only for coding blocks with a small delay time or memory usage (i.e., coding blocks with a small block size), dequantization and inverse transform processing of the transform block can be started after the secondary transform identifier is decoded, and for coding blocks with a large delay time or memory usage (i.e., coding blocks with a large block size), dequantization and inverse transform processing of the transform block can be started without waiting for the secondary transform identifier to be decoded.

[0438] Therefore, it is possible to suppress delays and increases in memory usage, that is, it is possible to suppress increases in the decoding load.

[0439] Note that the larger the block size, the greater the load of counting non-zero transform coefficients. Therefore, as described above, by setting an upper limit on the block size when a secondary transform is applied, it is possible to suppress the increase in the load of counting non-zero transform coefficients.

[0440] In addition, the conditional expression that references the number of non-zero transform coefficients may be deleted from the decoding / encoding condition of the secondary transform identifier. For example, the secondary transform identifier may be set regardless of the number of non-zero transform coefficients in the coding block. Also, for example, the secondary transform identifier may be analyzed regardless of the number of non-zero transform coefficients in the coding block. By doing so, it is possible to omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process and the decoding process.

[0441] For example, the threshold may be the maximum transform block size. For example, the secondary transform identifier may be set so that the secondary transform is performed when the horizontal block size is equal to or smaller than the maximum transform block size and the vertical block size is equal to or smaller than the maximum transform block size.

[0442] By doing so, the block size of the coding block to be subjected to the secondary transform can be limited to the maximum transform block size or less, and an increase in the decoding load can be suppressed as described above.

[0443] An example of syntax related to st_mode is shown in Figure 47. In this case, as shown in the second row from the top (the row with gray background), for stAllowed to be true, the horizontal coding block size (cbWidth) and the vertical coding block size (cbHeight) must each be less than or equal to the maximum transform block size (axTbSize) (&& (cbWidth <= maxTbSize && cbHeight <= maxTbSize)). In other words, if the horizontal coding block size (cbWidth) or the vertical coding block size (cbHeight) is greater than the maximum transform block size (axTbSize), the secondary transform identifier is not signaled. In this way, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients, the secondary transform identifier may be set based on the coding block size.

[0444] The threshold value is arbitrary and may be other than the maximum transform block size. Furthermore, the control using this threshold value may be any control as long as it is for limiting the block size to be subjected to secondary transform, and the specific method of comparison with the threshold value for this purpose is arbitrary and is not limited to the example of FIG. 47. For example, the block size in the longitudinal direction (the longer of the horizontal and vertical directions) of the coding block may be compared with the maximum transform block size. Furthermore, the area (i.e., the product of the horizontal block size and the vertical block size) may be used to compare with a threshold value (e.g., the square of the maximum transform block size). Furthermore, the comparison may be performed using logarithmic values ​​(log values).

[0445] In this case, the secondary translation identifier may be stored in the footer of the CU (that is, st_idx may be signaled after all TUs in the CU have been signaled).

[0446] <7-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 18. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0447] <Encoding parameter setting process flow> Furthermore, the image coding device 100 in this case performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Furthermore, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0448] An example of the flow of the encoding parameter setting process executed by the control unit 101 in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 48. In this case, the control unit 101 executes this encoding parameter setting process by applying the various methods described in <7-1. Signaling of Secondary Conversion Identifier>.

[0449] When the encoding parameter setting process starts, in step S501, the secondary transform identifier setting unit 151 of the control unit 101 sets the secondary transform identifier st_idx to be stored in the CU footer according to the block size. That is, the secondary transform identifier setting unit 151 sets the secondary transform identifier so that only CUs with a predetermined block size or less signal the secondary transform identifier (so that secondary transform is skipped for CUs with a block size larger than the predetermined block size). Therefore, as described above, it is possible to suppress delays and increases in memory usage in the decoding process. Furthermore, it is possible to suppress an increase in the load of counting non-zero transform coefficients. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0450] In this case, the secondary transform identifier setting unit 151 can set the secondary transform identifier regardless of the number of non-zero transform coefficients in the CU. This can omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the encoding process.

[0451] In step S502, the transform skip flag setting unit 152 sets a transform skip flag. In step S503, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier. In step S504, the context setting unit 154 sets a context. When the processing of step S504 ends, the encoding parameter setting processing ends, and the processing returns to FIG. 20.

[0452] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0453] <7-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 24. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0454] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0455] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 of Fig. 26 will be described with reference to the flowchart of Fig. 49. In this case, the decoding unit 212 executes this decoding process by applying the various methods described in <7-1. Signaling of Secondary Translation Identifier>.

[0456] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S521.

[0457] In step S522, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier of the CU footer (signaled after the transform coefficients of the CU) from the decoding result.

[0458] In this case, the secondary transform identifier is signaled only for coding blocks that are equal to or smaller than a threshold value (e.g., a maximum transform block size). Therefore, only for coding blocks with low delay time or memory usage (i.e., coding blocks with small block sizes), dequantization and inverse transform processing of the transform block can be started after the secondary transform identifier is decoded, and for coding blocks with large delay time or memory usage (i.e., coding blocks with large block sizes), dequantization and inverse transform processing of the transform block can be started without waiting for the secondary transform identifier to be decoded.

[0459] Therefore, it is possible to suppress an increase in delay and memory usage. Also, by setting an upper limit on the block size when the secondary transform is applied in this way, it is possible to suppress an increase in the load of counting non-zero transform coefficients. In other words, it is possible to suppress an increase in the decoding load.

[0460] The secondary transform identifier may be analyzed regardless of the number of non-zero transform coefficients in the coding block, which can omit the cumbersome counting of non-zero transform coefficients, thereby suppressing an increase in the load of the decoding process.

[0461] In step S523, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S524, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. In step S525, the context setting unit 254 sets a context. When the processing of step S525 ends, the decoding processing ends, and the processing returns to FIG. 26.

[0462] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0463] 8. Seventh Embodiment <8-1. Simplifying secondary conversion> <Controlling the size of the effective conversion area> When applying a secondary transform that zeros high-frequency primary transform coefficients as described in Non-Patent Document 9, i.e., when zeroing primary transform coefficients that are not changed by the RST transform matrix (primary transform coefficients other than the LFNST corners), the effective transform region size is derived based on the value of the secondary transform identifier. For example, if the transform block size is larger than 4x4, the effective transform region size is set to 4x4.

[0464] By doing so, it is possible to derive the effective transform domain size using a method corresponding to the secondary transform and use it to find the prefix part of the last coefficient. This makes it possible to suppress an increase in the code length. That is, it is possible to suppress an increase in the bin length of the last coefficient (typically, it is possible to reduce the bin length). That is, it is possible to suppress an increase in the amount of code (suppress a decrease in coding efficiency). Therefore, it is possible to suppress an increase in the load of the decoding process.

[0465] An example of the syntax for the transform block (residual_coding) in this case is shown in Figure 50. In the example of Figure 50, in the fourth and fifth lines from the top (gray lines), if a secondary transform is performed and the logarithm of the horizontal and vertical transform block sizes is greater than 2 (i.e., if the transform block is greater than 4x4), the logarithm of the horizontal effective transform area size (log2ZoTbWidth) is set to 2.

[0466] Similarly, in the 10th and 11th lines from the top (gray lines), if a secondary transform is performed and the logarithm of the horizontal and vertical transform block size is greater than 2 (i.e., if the transform block is greater than 4x4), the logarithm of the vertical effective transform area size (log2ZoTbHeight) is set to 2.

[0467] By doing so, it is possible to suppress an increase in the length of the bin string bins in the prefix part.

[0468] For example, suppose the last coefficient position (lastX, lastY) of the DC sub-block is (3, 3). In the case of the 16x16 TB shown in Fig. 9, the bin sequence of each prefix part (last_sig_coeff_x_prefix, last_sig_coeff_y_prefix) of the last coefficient position derived using Fig. 10 to Fig. 12 is "1110". In other words, the bin length is 4 bins.

[0469] On the other hand, when the effective transform area size is 4x4, the bin sequence of the prefix part is "111". In other words, the bin length is 3 bins.

[0470] In this way, when the transform block size is larger than 4x4, by setting the effective transform region size to 4x4, it is possible to shorten the length by one bin compared to the above example. Considering the X and Y directions, it is possible to reduce the number of bins by up to two. In this way, it is possible to suppress a decrease in coding efficiency. Furthermore, by suppressing an increase in the amount of code, it is possible to suppress an increase in the load of the decoding process.

[0471] <8-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is basically the same as that in the first embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Also, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0472] <Control unit> An example of the main configuration of the control unit 101 in this case is shown in Figure 51. As shown in Figure 51, the control unit 101 in this case has a secondary transform identifier setting unit 151 to a context setting unit 154, as well as an effective transform area size derivation unit 401 and a last coefficient position setting unit 402.

[0473] The effective transform area size derivation unit 401 performs processing related to derivation of the effective transform area size. For example, the effective transform area size derivation unit 401 derives, as a coding parameter, the effective transform area size (log2ZoTbWidth, log2ZoHeight), which is the area in which non-zero transform coefficients remain after the secondary transform that zeros high-frequency primary transform coefficients. In this case, the effective transform area size derivation unit 401 applies the method described in <8-1. Simplification of Secondary Transform>.

[0474] For example, when the transform block size is larger than 4x4, the effective transform region size derivation unit 401 sets the effective transform region size to 4x4. By doing so, it is possible to suppress an increase in the bin length of the prefix part of the last coefficient, and to suppress a decrease in coding efficiency. Therefore, it is possible to suppress an increase in the load of the decoding process.

[0475] The last coefficient position setting unit 402 performs processing related to setting of the last coefficient position. For example, the last coefficient position setting unit 402 binarizes each prefix part (last_sig_coeff_x_prefix, last_sig_coeff_y_prefix) of the last coefficient position (lastX, lastY) according to the table shown in Fig. 11 using the effective transform region size (log2ZoTbWidth, log2ZoHeight), and generates a bin string according to the table shown in Fig. 12. The last coefficient position setting unit 402 supplies this bin string to the encoding unit 115, which encodes it.

[0476] <Encoding parameter setting process flow> In this case, the image coding device 100 performs basically the same processing as in the first embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Moreover, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0477] An example of the flow of the encoding parameter setting process executed by the control unit 101 in step S103 of Fig. 20 in this case will be described with reference to the flowchart of Fig. 52. In this case, the control unit 101 executes this encoding parameter setting process by applying the technique described in <8-1. Simplification of secondary conversion>.

[0478] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the CU footer in step S541.

[0479] In step S542, the transform skip flag setting unit 152 sets a transform skip flag. In step S543, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier. In step S544, the context setting unit 154 sets a context.

[0480] In step S545, the valid transform region size derivation unit 401 derives the valid transform region size based on the secondary transform identifier of the CU. For example, in the case of a secondary transform that zeros high-frequency primary transform coefficients, if the transform block size is larger than 4x4, the valid transform region size derivation unit 401 sets the valid transform region size to 4x4.

[0481] In step S546, the last coefficient position setting unit 402 derives the last coefficient position based on the effective transform region size.

[0482] In step S547, the last coefficient position setting unit 402 binarizes the prefix part of the last coefficient position and derives a bin sequence.

[0483] When the process of step S547 ends, the encoding parameter setting process ends, and the process returns to FIG.

[0484] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0485] By performing the encoding parameter setting process in this manner, it is possible to suppress a decrease in encoding efficiency.

[0486] <8-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is basically the same as that in the first embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Also, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0487] <Decryption section> An example of the main configuration of the decoding unit 212 in this case is shown in Fig. 53. As shown in Fig. 53, the decoding unit 212 in this case has a decoding unit 241 and an analyzing unit 242, similar to the first embodiment.

[0488] The analysis unit 242 includes a last coefficient position analysis unit 421 and a valid transformation area size analysis unit 422 in addition to the components from the secondary transformation identifier analysis unit 251 to the context setting unit 254 .

[0489] The last coefficient position analysis unit 421 performs processing related to analysis of the last coefficient position. For example, the last coefficient position analysis unit 421 de-binarizes (converts) the bin string of the prefix part of the last coefficient position.

[0490] The effective transform area size analysis unit 422 performs processing related to analysis of the effective transform area size. For example, the effective transform area size analysis unit 422 calculates the effective transform area size based on the last coefficient position obtained by the last coefficient position analysis unit 421. The effective transform area size analysis unit 422 supplies the calculated effective transform area size to the inverse orthogonal transform unit 214. The inverse secondary transform unit 271 of the inverse orthogonal transform unit 214 uses the effective transform area size to perform an inverse secondary transform corresponding to a secondary transform that zeros out primary transform coefficients in the high frequency range (FIG. 9).

[0491] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the first embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0492] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 in FIG. 26 will be described with reference to the flowchart in FIG.

[0493] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S561.

[0494] In step S562, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier of the CU footer (signaled after the transform coefficients of the CU) from the decoding result.

[0495] In step S563, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S564, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. In step S565, the context setting unit 254 sets a context.

[0496] In step S566, the last coefficient position analysis unit 421 de-binarizes (converts) the bin string in the prefix part of the last coefficient position.

[0497] In step S567, the effective transform region size analysis unit 422 calculates the effective transform region size based on the obtained last coefficient position.

[0498] When the process of step S567 ends, the decoding process ends and the process returns to FIG.

[0499] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0500] 9. Eighth Embodiment <9-1. Simplifying secondary conversion> <Controlling the size of the effective conversion area> The present technology described in the seventh embodiment may be combined with the second embodiment. That is, a secondary transform identifier may be signaled for each TU. That is, even when the present technology described in the seventh embodiment is applied to the second embodiment, in the secondary transform that zeros out the primary transform coefficients in the high frequency range, if the transform block size is larger than 4x4, the effective transform region size may be set to 4x4.

[0501] By doing so, as in the seventh embodiment, it is possible to suppress an increase in the bin length of the last coefficient and a decrease in coding efficiency, and therefore it is possible to suppress an increase in the load of the decoding process.

[0502] <9-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the seventh embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 51. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0503] <Encoding parameter setting process flow> In this case, the image coding device 100 performs basically the same processing as in the seventh embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Moreover, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0504] An example of the flow of the encoding parameter setting process executed by the control unit 101 in this case in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 55. In this case, the control unit 101 executes this encoding parameter setting process by applying the technique described in <9-1. Simplification of secondary conversion>.

[0505] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TU footer in step S601.

[0506] In step S602, the transform skip flag setting unit 152 sets a transform skip flag. In step S603, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier. In step S604, the context setting unit 154 sets a context.

[0507] In step S605, the valid transform region size derivation unit 401 derives the valid transform region size based on the secondary transform identifier of the TU. For example, in the case of a secondary transform that zeros high-frequency primary transform coefficients, if the transform block size is larger than 4x4, the valid transform region size derivation unit 401 sets the valid transform region size to 4x4.

[0508] In step S606, the last coefficient position setting unit 402 derives the last coefficient position based on the effective transform region size. In step S607, the last coefficient position setting unit 402 binarizes the prefix part of the last coefficient position and derives a bin sequence. When the processing of step S607 ends, the encoding parameter setting process ends, and the process returns to FIG. 20.

[0509] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0510] By performing the encoding parameter setting process in this manner, it is possible to suppress a decrease in encoding efficiency.

[0511] <9-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the seventh embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 53. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0512] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the seventh embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0513] An example of the flow of the decoding process executed by the decoding unit 212 in step S202 in FIG. 26 in this case will be described with reference to the flowchart in FIG.

[0514] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S621.

[0515] In step S622, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier in the TU footer (signaled after the transform coefficients of the TU) from the decoding result.

[0516] In step S623, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S624, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. In step S625, the context setting unit 254 sets a context.

[0517] In step S626, the last coefficient position analysis unit 421 de-binarizes (converts to multi-value) the bin string in the prefix part of the last coefficient position. In step S627, the effective transform area size analysis unit 422 calculates the effective transform area size based on the obtained last coefficient position. When the process of step S627 ends, the decoding process ends and the process returns to FIG. 26.

[0518] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0519] 10. Ninth Embodiment <10-1. Simplifying secondary conversion> <Controlling the size of the effective conversion area> The present technology described in the seventh embodiment may be combined with the fourth embodiment. That is, a secondary transform identifier may be signaled for each TB. That is, even when the present technology described in the seventh embodiment is applied to the fourth embodiment, in the secondary transform that zeros the primary transform coefficients in the high frequency range, if the transform block size is larger than 4x4, the effective transform region size may be set to 4x4.

[0520] By doing so, as in the seventh embodiment, it is possible to suppress an increase in the bin length of the last coefficient and a decrease in coding efficiency, and therefore it is possible to suppress an increase in the load of the decoding process.

[0521] An example of the syntax for the transform block (residual_coding) in this case is shown in Figure 57. In the example of Figure 57, in the fourth and fifth lines from the top (gray lines), a secondary transform is performed, and if the logarithm of the horizontal and vertical transform block sizes is greater than 2 (i.e., if the transform block is greater than 4x4), the logarithm of the horizontal effective transform area size (log2ZoTbWidth) is set to 2. In this case, the secondary transform is determined for each component (each cIdx), that is, for each transform block.

[0522] Similarly, in the 10th and 11th lines from the top (gray lines), if a secondary transform is performed and the logarithm of the horizontal and vertical transform block sizes is greater than 2 (i.e., if the transform block is greater than 4x4), the logarithm of the vertical effective transform area size (log2ZoTbHeight) is set to 2. In this case, the secondary transform determination is performed for each component (each cIdx), that is, for each transform block.

[0523] By doing so, it is possible to suppress an increase in the length of the bin string bins in the prefix part.

[0524] <10-2. Encoding side> <Configuration> Next, the encoding side will be described. The configuration of the encoding side in this case is the same as that in the seventh embodiment. That is, the image encoding device 100 in this case has the same configuration as that described with reference to Fig. 17. Furthermore, the control unit 101 in this case has the same configuration as that described with reference to Fig. 51. Furthermore, the orthogonal transform unit 113 in this case has the same configuration as that described with reference to Fig. 19.

[0525] <Encoding parameter setting process flow> In this case, the image coding device 100 performs basically the same processing as in the seventh embodiment. That is, the image coding processing executed by the image coding device 100 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 20. Moreover, the orthogonal transform processing executed by the orthogonal transform unit 113 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 22.

[0526] An example of the flow of the encoding parameter setting process executed by the control unit 101 in this case in step S103 of Fig. 20 will be described with reference to the flowchart of Fig. 58. In this case, the control unit 101 executes this encoding parameter setting process by applying the technique described in <10-1. Simplification of secondary conversion>.

[0527] When the encoding parameter setting process starts, the secondary transformation identifier setting unit 151 of the control unit 101 sets the secondary transformation identifier st_idx to be stored in the TB footer in step S641.

[0528] In step S642, the transform skip flag setting unit 152 sets a transform skip flag. In step S643, the adaptive orthogonal transform identifier setting unit 153 sets an adaptive orthogonal transform identifier. In step S644, the context setting unit 154 sets a context.

[0529] In step S645, the valid transform region size derivation unit 401 derives the valid transform region size based on the secondary transform identifier of the TB. For example, in the case of a secondary transform that zeros high-frequency primary transform coefficients, if the transform block size is larger than 4x4, the valid transform region size derivation unit 401 sets the valid transform region size to 4x4.

[0530] In step S646, the last coefficient position setting unit 402 derives the last coefficient position based on the effective transform region size. In step S647, the last coefficient position setting unit 402 binarizes the prefix part of the last coefficient position and derives a bin sequence. When the processing of step S647 ends, the encoding parameter setting process ends, and the process returns to FIG. 20.

[0531] In this encoding parameter setting process, a step of generating other arbitrary encoding parameters may be added.

[0532] By performing the encoding parameter setting process in this manner, it is possible to suppress a decrease in encoding efficiency.

[0533] <10-3. Decryption side> <Configuration> Next, the decoding side will be described. The configuration of the decoding side in this case is the same as that in the seventh embodiment. That is, the image decoding device 200 in this case has the same configuration as that described with reference to Fig. 23. Moreover, the decoding unit 212 in this case has the same configuration as that described with reference to Fig. 53. Furthermore, the inverse orthogonal transform unit 214 in this case has the same configuration as that described with reference to Fig. 25.

[0534] <Decryption process flow> Furthermore, the image decoding device 200 in this case performs basically the same processing as in the seventh embodiment. That is, the image decoding processing executed by the image decoding device 200 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 26. Furthermore, the inverse orthogonal transform processing executed by the inverse orthogonal transform unit 214 in this case is performed in the same flow as that described with reference to the flowchart in Fig. 28.

[0535] An example of the flow of the decoding process executed by the decoding unit 212 in this case in step S202 in FIG. 26 will be described with reference to the flowchart in FIG.

[0536] When the decoding process starts, the decoding unit 241 decodes the coded data (bit stream) in the accumulation buffer 211 in step S661.

[0537] In step S662, the secondary transform identifier analysis unit 251 of the analysis unit 242 analyzes the secondary transform identifier of the TB footer (signaled after the transform coefficients of the TB) from the decoding result.

[0538] In step S663, the transform skip flag analysis unit 252 analyzes the transform skip flag. In step S664, the adaptive orthogonal transform identifier analysis unit 253 analyzes the adaptive orthogonal transform identifier. In step S665, the context setting unit 254 sets a context.

[0539] In step S666, the last coefficient position analysis unit 421 de-binarizes (converts to multi-value) the bin string in the prefix part of the last coefficient position. In step S667, the effective transform area size analysis unit 422 calculates the effective transform area size based on the obtained last coefficient position. When the process of step S667 ends, the decoding process ends and the process returns to FIG. 26.

[0540] Note that the decoding process may include an additional step of analyzing any other coding parameters.

[0541] <11. Notes> <Combination> The present technology described in each of the above embodiments can be applied in combination with the present technology described in any other embodiment, as long as no contradiction occurs.

[0542] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0543] FIG. 60 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0544] In a computer 800 shown in FIG. 60, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0545] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0546] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0547] In a computer configured as above, the CPU 801 performs the above-described series of processes by, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data necessary for the CPU 801 to execute various processes as appropriate.

[0548] The program executed by the computer can be applied by recording it on removable media 821 such as package media, for example. In this case, the program can be installed in storage unit 813 via input / output interface 810 by inserting removable media 821 into drive 815.

[0549] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0550] Alternatively, this program can be installed in advance in the ROM 802 or the storage unit 813 .

[0551] <Unit of information and processing> The data units in which the various pieces of information described above are set and the data units targeted by the various processes are each arbitrary and are not limited to the above examples. For example, these pieces of information and processes may be set for each TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), sub-block, block, tile, slice, picture, sequence, or component, or may target data of these data units. Of course, these data units may be set for each piece of information or process, and the data units for all pieces of information and processes do not need to be unified. Note that the storage location of these pieces of information is arbitrary and may be stored in the headers or parameter sets of the above-mentioned data units, or may be stored in multiple locations.

[0552] <Control information> Control information related to the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether or not to permit (or prohibit) application of the above-described present technology may be transmitted. Also, for example, control information (e.g., present_flag) that indicates a target to which the above-described present technology is to be applied (or a target to which it is not to be applied) may be transmitted. For example, control information that specifies a block size (upper or lower limit, or both), frame, component, or layer to which the present technology is to be applied (or permitted or prohibited to be applied) may be transmitted.

[0553] <Applicable targets of this technology> This technology can be applied to any image encoding / decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding / decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. Furthermore, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0554] This technology can also be applied to a multi-viewpoint image encoding / decoding system that encodes and decodes multi-viewpoint images that include images from multiple views. In this case, this technology can be applied to the encoding and decoding of each view.

[0555] Furthermore, this technology can be applied to a layered image coding (scalable coding) and decoding system that encodes and decodes layered images with multiple layers to provide scalability for predetermined parameters. In this case, this technology can be applied to the encoding and decoding of each layer.

[0556] Furthermore, although the image encoding device 100 and the image decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0557] For example, this technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) used in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communications, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0558] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0559] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0560] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0561] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0562] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0563] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0564] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0565] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0566] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0567] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0568] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0569] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0570] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0571] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0572] The present technology can also be configured as follows. (1) a setting unit that sets a secondary conversion identifier so that secondary conversion is performed only when information about the block size is equal to or smaller than a predetermined threshold; a transform unit that performs a secondary transform on coefficient data derived from image data based on the secondary transform identifier set by the setting unit; an encoding unit that encodes the secondary translation identifier set by the setting unit and generates a bitstream; An image processing device comprising: (2) The threshold is the maximum transform block size. The image processing device according to (1). (3) the information about the block size is a block size in the horizontal direction of the block to be processed and a block size in the vertical direction of the block to be processed, The setting unit sets the secondary transform identifier so as to perform the secondary transform when the horizontal block size and the vertical block size are equal to or smaller than the maximum transform block size. (2) An image processing device according to the present invention. (4) The setting unit sets the secondary transform identifier so as to perform the secondary transform when the horizontal block size and the vertical block size are equal to or smaller than the maximum transform block size, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients. (3) An image processing device according to the present invention. (5) The setting unit sets the secondary translation identifier to be stored in a footer of the coding block. (4) An image processing device according to (4). (6) setting a secondary conversion identifier so that secondary conversion is performed only when the information about the block size is equal to or less than a predetermined threshold; performing a secondary transformation on coefficient data derived from image data based on the set secondary transformation identifier; The set secondary translation identifier is encoded to generate a bitstream. Image processing methods.

[0573] (7) a secondary translation identifier setting unit that sets a secondary translation identifier to be stored in a portion other than the footer of the coding block; a transform unit that performs a secondary transform on coefficient data derived from image data based on the secondary transform identifier set by the secondary transform identifier setting unit; an encoding unit that encodes the secondary translation identifier set by the secondary translation identifier setting unit and generates a bitstream; An image processing device comprising: (8) The secondary translation identifier setting unit sets the secondary translation identifier to be stored in a header of the coding block. (7) An image processing device according to (7). (9) The secondary transform identifier setting unit sets the secondary transform identifier based on a flag indicating whether a residual PCM coding mode is applied, instead of setting the secondary transform identifier based on the number of non-zero transform coefficients. An image processing device according to (7) or (8). (10) Further comprising a context setting unit that sets a context based on the tree type. An image processing device according to any one of (7) to (9). (11) The secondary translation identifier setting unit sets the secondary translation identifier to be stored in the translation unit. An image processing device according to any one of (7) to (10). (12) The secondary translation identifier setting unit sets the secondary translation identifier to be stored in a translation block. An image processing device according to any one of (7) to (10). (13) Further comprising a context setting unit that sets a context based on a component An image processing device according to any one of (7) to (12). (14) The transform block to which the secondary transform that zeros the primary transform coefficients in the high frequency range is applied further includes an effective transform region size derivation unit that sets the effective transform region size to 4x4 when the transform block size is larger than 4x4. An image processing device according to any one of (7) to (13). (15) setting a secondary transformation identifier to be stored outside the footer of the coding block; and performing a secondary transformation on coefficient data derived from image data based on the set secondary transformation identifier; The set secondary translation identifier is encoded to generate a bitstream. Image processing methods. [Explanation of symbols]

[0574] 100 Image encoding device, 101 Control unit, 113 Orthogonal transform unit, 115 Encoding unit, 118 Inverse orthogonal transform unit, 151 Secondary transform identifier setting unit, 152 Transform skip flag setting unit, 153 Adaptive orthogonal transform identifier setting unit, 154 Context setting unit, 172 Secondary transform unit, 200 Image decoding device, 212 Decoding unit, 214 Inverse orthogonal transform unit, 242 Analysis unit, 251 Secondary transform identifier analysis unit, 252 Transform skip flag analysis unit, 253 Adaptive orthogonal transform identifier analysis unit, 254 Context setting unit, 271 Inverse secondary transform unit, 401 Effective transform region size derivation unit, 402 Last coefficient position setting unit, 421 Last coefficient position analysis unit, 422 Effective transform region size analysis unit< / ctxinc>

Claims

1. a setting unit that sets an index of a context of a secondary translation identifier based only on whether the tree type is a single tree; a coding unit that arithmetically codes the secondary translation identifier according to the index of the context set by the setting unit; An image processing device comprising:

2. The setting unit sets a context index for a first bin of the secondary translation identifier based only on whether the tree type is a single tree. The image processing device according to claim 1 .

3. indexing the context of the secondary translation identifier based solely on whether the tree type is single-tree; Arithmetically encoding the secondary transformation identifier according to the index of the set context. Image processing methods.

4. a setting unit that sets an index of a context of a secondary translation identifier based only on whether the tree type is a single tree; a decoding unit that arithmetically decodes the secondary transformation identifier according to the index of the context set by the setting unit; An image processing device comprising:

5. indexing the context of the secondary translation identifier based solely on whether the tree type is single-tree; Arithmetically decoding the secondary transformation identifier according to the set index of the context. Image processing methods.

Images