Image processing device and method

Abstract

To enable suppression of reduction in encoding efficiency.SOLUTION: In the case that a conversion skip flag of at least one component among valid components in image encoding is true, encoding of secondary conversion control information being control information related to secondary conversion is omitted, and in the case that conversion skip flags of all the components are false, the secondary conversion control information is encoded. The present disclosure can be applied to, for example, an image processing device, an image encoding device, an image decoding device, a transmission device, a reception device, a transmission / reception device, an information processing device, an imaging device, a reproduction device, an electronic device, an image processing method, an information processing method, or the like.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method capable of suppressing a decrease in encoding efficiency. [Background technology]

[0002] Conventionally, a coding method has been proposed in which a prediction residual of a video is derived, coefficient transformed, quantized, and coded (for example, Non-Patent Document 1). In VVC (Versatile Video Coding) described in Non-Patent Document 1, a coding tool is provided that performs LFNST (Low Frequency Non-Separable Transform) as a secondary transform on transform coefficients after primary transform, and further improves energy compaction. Then, secondary transform control information, which is control information related to this secondary transform, is signaled (i.e., the secondary transform control information is coded and transmitted to the decoding side).

[0003] As this secondary transform control information, for example, an LFNST identifier (lfnst_idx) indicating the LFNST mode is signaled. This LFNST identifier is signaled according to the values ​​of the tree type (treeType) and the transform skip flag (transform_skip_flag). The transform skip flag is flag information indicating whether or not to skip (omit) coefficient transform including the above-mentioned primary transform and secondary transform in image coding. Note that this transform skip flag also indicates whether or not to skip (omit) the inverse process (inverse transform) of the coefficient transform in image decoding. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, "Versatile Video Coding (Draft 7)", JVET-P2001-vE, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Geneva, CH, 1-11 Oct 2019 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the case of the method described in Non-Patent Document 1, even if the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true (for example, 1), the LFNST identifier is signaled, which may be redundant, and thus may reduce the coding efficiency.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]

[0007] According to one aspect of the present technology, there is provided an image processing device comprising: a derivation unit that derives a variable indicating that the transform skip is not applied to all components by referring to the transform skip flag and the tree type, and omits coding of a secondary transform identifier that is an identifier indicating a type of secondary transform when the variable derived by the derivation unit indicates false; Encoding section and The image processing device includes:

[0008] An image processing method according to one aspect of the present technology includes: The method includes: deriving a variable indicating that the transform skip is not applied to all components by referring to the transform skip flag and the tree type; and omitting coding for a secondary transform identifier, which is an identifier indicating a type of the secondary transform, when the derived variable indicates false. An image processing method.

[0009] An image processing device according to another aspect of the present technology includes: a derivation unit that derives a variable indicating that a transform skip is not applied to all components by referring to the transform skip flag and the tree type; and a decoding unit that omits decoding of a secondary transform identifier that is an identifier indicating a type of secondary transform when the variable derived by the derivation unit indicates false. The image processing device is provided with:

[0010] An image processing method according to another aspect of the present technology includes: The method includes: deriving a variable indicating that a transform skip is not applied to all components by referring to the transform skip flag and the tree type; and omitting decoding of a secondary transform identifier, which is an identifier indicating a type of the secondary transform, when the derived variable indicates false. An image processing method.

[0011] In one aspect of the present technology, an image processing device and method include: The transform skip flag and the tree type are referenced to derive a variable indicating that the transform skip is not applied to all components, and if the derived variable indicates false, encoding of the secondary transform identifier, which is an identifier indicating the type of secondary transform, is omitted.

[0012] In another aspect of the present technology, an image processing device and a method thereof include: The transform skip flag and the tree type are referenced to derive a variable indicating that the transform skip is not applied to all components, and if the derived variable indicates false, decoding of the secondary transform identifier, which is an identifier indicating the type of secondary transform, is omitted. [Brief description of the drawings]

[0013] [Figure 1] FIG. 13 is a diagram illustrating an example of a syntax table regarding LFNST identifiers. [Diagram 2] FIG. 13 is a diagram showing an example of an LFNST identifier based on a tree type and a transformation skip flag. [Diagram 3] FIG. 13 is a diagram showing an example of an LFNST identifier based on a tree type and a transformation skip flag. [Figure 4] FIG. 13 is a diagram illustrating an example of a syntax table regarding LFNST identifiers. [Diagram 5] FIG. 13 is a diagram showing an example of the encoding condition “condition.” [Figure 6] FIG. 13 is a diagram illustrating an example of a syntax table regarding LFNST identifiers. [Figure 7] FIG. 13 is a diagram showing an example of the encoding condition “condition.” [Figure 8] FIG. 13 is a diagram showing an example of an LFNST identifier based on a tree type and a transformation skip flag. [Figure 9] FIG. 13 is a diagram showing an example of an LFNST identifier based on a tree type and a transformation skip flag. [Figure 10] FIG. 13 is a diagram showing an example of an LFNST identifier based on a tree type and a transformation skip flag. [Figure 11] FIG. 2 is a block diagram showing an example of the main configuration of an encoding device. [Figure 12] 13 is a flowchart showing an example of the flow of an encoding process. [Figure 13] FIG. 2 is a block diagram showing an example of the main configuration of a decoding device. [Figure 14] 13 is a flowchart showing an example of the flow of a decoding process. [Figure 15] FIG. 1 is a block diagram showing an example of the main configuration of an image encoding device. [Figure 16] 13 is a flowchart showing an example of the flow of an image encoding process. [Figure 17] FIG. 2 is a block diagram showing an example of the main configuration of an image decoding device. [Figure 18] 13 is a flowchart showing an example of the flow of an image decoding process. [Figure 19] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. LFNST Identifier Signaling Pattern 2. First embodiment (encoding device) 3. Second embodiment (decoding device) 4. Third embodiment (image encoding device) 5. Fourth embodiment (image decoding device) 6. Notes

[0015] <1. LFNST Identifier Signaling Pattern> <References supporting technical content and technical terms> The scope of disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application, and the contents of other documents referenced in the following non-patent documents.

[0016] Non-patent document 1: (mentioned above) Non-patent document 2: Jianle Chen, Yan Ye, Seung Hwan Kim, "Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7)", JVET-P2002-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Geneva, CH, 1-11 Oct. 2019 Non-patent document 3: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-Patent Document 4: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0017] In other words, the contents described in the above non-patent documents are also the basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above non-patent documents are not directly described in the examples, they are considered to be within the disclosure scope of the present technology and to satisfy the support requirements of the claims. Similarly, for example, technical terms such as parsing, syntax, and semantics are also considered to be within the disclosure scope of the present technology and to satisfy the support requirements of the claims, even if they are not directly described in the examples.

[0018] In addition, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) indicates any partial region in a picture, and its size, shape, characteristics, etc. are not limited, unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), subblock, macroblock, tile, or slice described in the above-mentioned non-patent document.

[0019] In addition, when specifying the size of such a block, the block size may be specified indirectly in addition to directly. For example, the block size may be specified using identification information for identifying the size. For example, the block size may be specified by a ratio or difference with respect to the size of a reference block (for example, LCU or SCU). For example, when transmitting information specifying a block size as a syntax element or the like, the information may be used to indirectly specify the size as described above. In this way, the amount of information can be reduced, and coding efficiency may be improved. The specification of the block size also includes specification of a range of block sizes (for example, specification of a range of allowable block sizes, etc.).

[0020] Also, in this specification, encoding includes not only the entire process of converting an image into a bitstream, but also some processes. For example, it includes not only processes including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also processes collectively referred to as quantization and arithmetic coding, processes including prediction processing, quantization, and arithmetic coding, and the like. Similarly, decoding includes not only the entire process of converting a bitstream into an image, but also some processes. For example, it includes not only processes including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also processes including inverse arithmetic decoding and inverse quantization, processes including inverse arithmetic decoding, inverse quantization, and prediction processing, and the like.

[0021] <LFNST identifier> In the VVC (Versatile Video Coding) described in Non-Patent Document 1, for the transform coefficients after primary transformation, LFNST (Low Frequency Non-Separable Transform) was performed as a secondary transform, and there was an encoding tool to further improve energy compaction. And the secondary transform control information, which is control information regarding this secondary transform, was signaled (that is, the secondary transform control information was encoded in the encoder and transmitted to the decoder side).

[0022] As this secondary transform control information, for example, an LFNST identifier (lfnst_idx), which is an identifier indicating the mode of LFNST, was signaled. When the value of the LFNST identifier (lfnst_idx) is "0", it indicates that the secondary transform (LFNST) is skipped (omitted). Also, when the value of the LFNST identifier (lfnst_idx) is "1", it indicates that the secondary transform by the first method is applied. Furthermore, when the value of the LFNST identifier (lfnst_idx) is "2", it indicates that the secondary transform by the second method is applied. Note that, as will be described later, the signaling of this LFNST identifier (lfnst_idx) can also be skipped. When the signaling of the LFNST identifier (lfnst_idx) is skipped, the decoder side (decoder) estimates its value.

[0023] Based on the LFNST identifier (or estimated LFNST identifier) ​​signaled from the encoding side (encoder), the decoder can perform an inverse secondary transform, which is the inverse process of the secondary transform performed by the encoder, in a mode corresponding to the secondary transform. In other words, the secondary transform control information can also be considered as control information related to the inverse secondary transform. The LFNST identifier can also be considered as an identifier indicating the mode of the inverse process of LFNST performed as the inverse secondary transform.

[0024] By signaling the LFNST identifier as described above, the encoding side (encoder) can control the inverse secondary transform at the decoding side (decoder), i.e., the encoder can make the decoder perform the inverse secondary transform in a mode corresponding to the secondary transform at the time of encoding.

[0025] An example of a syntax table related to this LFNST identifier in VVC described in Non-Patent Document 1 is shown in Fig. 1. As shown in Fig. 1, when the conditions in the seventh and eighth rows from the top of this syntax table are satisfied, the LFNST identifier (lfnst_idx) shown in the ninth row from the top is signaled. That is, this LFNST identifier is signaled according to the values ​​of the tree type (treeType) and the transform skip flag (transform_skip_flag). The transform skip flag is flag information indicating whether or not to skip (omit) coefficient transform including the above-mentioned primary transform and secondary transform in image coding. Note that this transform skip flag also indicates whether or not to skip (omit) the inverse process (inverse transform) of the coefficient transform in image decoding.

[0026] FIG. 2 shows an example of the correspondence between the combination of the tree type and the transform skip flag and the signaling of the LFNST identifier. In FIG. 2, there are three types of tree types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. In addition, transform_skip_flag[0] indicates the transform skip flag of the luminance (Y) component. transform_skip_flag[1] indicates the transform skip flag of the chrominance (Cb) component. transform_skip_flag[2] indicates the transform skip flag of the chrominance (Cr) component. The "0" and "1" in the column of each transform skip flag in the table shown in FIG. 2 indicate the value of the transform skip flag. In this example, the value "1" indicates true, and the value "0" indicates false. Note that the transform skip flag of an invalid component (Y, Cb, Cr) of each tree type is not signaled. In FIG. 2, the value of the transform skip flag of such an invalid component is indicated by "-".

[0027] In this example, if the value of transform_skip_flag[0] is "0" or the tree type is DUAL_TREE_CHROMA, the LFNST identifier may be signaled. Otherwise, the signaling of the LFNST identifier is skipped. In this case, the decoder infers the value of the LFNST identifier to be "0" (Infer to 0).

[0028] However, as shown in the bottom row of the table in FIG. 2, the LFNST identifier was also signaled when the tree type was dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb and Cr components of the image were true (e.g., 1). As described above, when the transform skip flag is true, the coefficient transform (inverse coefficient transform) is skipped, and therefore the secondary transform (inverse secondary transform) is also skipped. When the tree type is dual tree chroma (DUAL_TREE_CHROMA), the valid components are Cb and Cr, so that if the transform skip flags of these components are true, the secondary transform (inverse secondary transform) is always skipped. Therefore, the signaling of the LFNST identifier may be redundant. In other words, there may be a risk of unnecessarily reducing the coding efficiency.

[0029] <Method 1> Therefore, if the conversion skip flag of at least one valid component is true, the signaling of the LFNST identifier is skipped (method 1).

[0030] For example, in an image processing method, if the transform skip flag of at least one component among the valid components in image encoding is true, encoding of secondary transform control information, which is control information related to secondary transformation, is omitted, and if the transform skip flags of all components are false, the secondary transform control information is encoded.

[0031] For example, in an image processing device, if the conversion skip flag of at least one component among the valid components in image encoding is true, encoding of secondary conversion control information, which is control information related to secondary conversion, is omitted, and if the conversion skip flags of all components are false, an encoding unit is provided that encodes the secondary conversion control information.

[0032] For example, in an image processing method, if the conversion skip flag of at least one component among the valid components in image decoding is true, decoding of secondary conversion control information, which is control information related to secondary conversion, is omitted, and the value of the secondary conversion control information is presumed to be a predetermined value indicating skipping of secondary conversion, and if the conversion skip flags of all components are false, the bitstream is decoded to generate secondary conversion control information.

[0033] For example, in an image processing device, if the conversion skip flag of at least one component among the valid components in image decoding is true, decoding of secondary conversion control information, which is control information related to secondary conversion, is omitted, and the value of the secondary conversion control information is estimated to be a predetermined value indicating skipping of secondary conversion, and if the conversion skip flags of all components are false, a decoding unit is provided that decodes the bitstream and generates secondary conversion control information.

[0034] In addition, the decoding of the secondary conversion control information means decoding a bit stream (also called coded data) generated by coding the secondary conversion control information to generate the secondary conversion control information. In addition, the predetermined value indicating skipping the secondary conversion is arbitrary. For example, it may be "0".

[0035] The secondary conversion control information may be any control information related to the secondary conversion, and may include, for example, a secondary conversion identifier that is an identifier indicating the type of secondary conversion (and inverse secondary conversion).

[0036] The secondary transform may be any coefficient transform, for example, LFNST (Low Frequency Non-Separable Transform). In this case, the inverse secondary transform may be an inverse process of LFNST. In this case, the secondary transform identifier is an LFNST identifier (lfnst_idx). The predetermined value indicating skipping of the secondary transform may be "0" (lfnst_idx == 0).

[0037] An example of the correspondence between combinations of tree types and transform skip flags and the signaling of LFNST identifiers when Method 1 is applied is shown in Fig. 3. Fig. 3 is a table corresponding to the table in Fig. 2. The secondary transform is LFNST, and an LFNST identifier is signaled as the secondary transform information. Also, the predetermined value indicating skipping of the secondary transform is assumed to be "0" (lfnst_idx == 0).

[0038] As in FIG. 2, there are three types of tree types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Also, transform_skip_flag[0] indicates the transform skip flag of the luminance (Y) component. transform_skip_flag[1] indicates the transform skip flag of the chrominance (Cb) component. transform_skip_flag[2] indicates the transform skip flag of the chrominance (Cr) component. The "0" and "1" in each transform skip flag column indicate the value of that transform skip flag. In this example, a value "1" indicates true, and a value "0" indicates false. Note that the transform skip flags (transform skip flags indicated by "-") of invalid components (Y, Cb, Cr) of each tree type are not signaled.

[0039] In this example, regardless of the tree type, the LFNST identifier is signaled only if the transform skip flag of all valid components is false (value "0"). If the transform skip flag of at least one valid component is true (value "1"), the signaling of the LFNST identifier is skipped (omitted). In that case, the decoding side (decoder) infers the value of the LFNST identifier to be "0" (i.e., a value indicating skipping of the secondary transform).

[0040] By doing this, when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb component and the Cr component of the image are true, it is possible to skip signaling of the LFNST identifier (the bottom row of the table in FIG. 3). Even if signaling of the LFNST identifier is skipped in this way, the decoder can skip the inverse secondary transform based on the LFNST identifier. In other words, the decoder can obtain an LFNST identifier whose value is not inconsistent with the signaled transform skip flag, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse secondary transform in the decoder.

[0041] That is, by applying the above-described method 1, it is possible to reduce redundancy in signaling of the LFNST identifier, and to prevent a decrease in coding efficiency.

[0042] For example, when the tree type of image coding is a single tree (SINGLE_TREE), Y, Cb, and Cr are valid components. Therefore, in this case, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is true, the coding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). Note that the Y component transform skip flag is the transform skip flag of the Y component (transform_skip_flag[0]). The Cb component transform skip flag is the transform skip flag of the Cb component (transform_skip_flag[1]). The Cr component transform skip flag is the transform skip flag of the Cr component (transform_skip_flag[2]).

[0043] On the other hand, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, it is possible that the secondary transform control information is coded, that is, the LFNST identifier (lfnst_idx) is signaled.

[0044] Also, for example, when the tree type of image decoding is a single tree, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is true, decoding related to the secondary transform control information is skipped. Then, it is estimated that the value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, it is estimated that the value of the LFNST identifier is "0" (Infer to 0). On the other hand, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the bit stream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0045] In the example of FIG. 2, when the tree type is a single tree and the Y component transform skip flag is false (transform_skip_flag[0] == 0), the LFNST identifier (lfnst_idx) is signaled.

[0046] On the other hand, in the example of Fig. 3, if the tree type is a single tree, and the Y component transform skip flag is false, and further, if the Cb component transform skip flag and the Cr component transform skip flag are not false (transform_skip_flag[1] == 0 && transform_skip_flag[2] == 0), the signaling of the LFNST identifier (lfnst_idx) is skipped. Therefore, it is possible to suppress an increase in the capacity of the buffer in the decoder. Also, it is possible to suppress an increase in the delay of the decoding process.

[0047] For example, when the tree type of image coding is dual tree luma (DUAL_TREE_LUMA), luminance Y is a valid component. Therefore, in this case, if the Y component transform skip flag is true, the coding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if the Y component transform skip flag is false, it is possible that the secondary transform control information is coded. That is, the LFNST identifier (lfnst_idx) is signaled.

[0048] Also, for example, when the tree type of image decoding is dual tree luma, if the Y component transform skip flag is true, decoding related to the secondary transform control information is skipped. Then, the value of the secondary transform control information is estimated to be a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if the Y component transform skip flag is false, the bitstream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0049] For example, when the tree type of image coding is dual tree chroma (DUAL_TREE_CHROMA), chrominance Cb and Cr are valid components. Therefore, in this case, if at least one of the Cb component transform skip flag and the Cr component transform skip flag is true, coding of the secondary transform control information is skipped. That is, in this case, signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if the Cb component transform skip flag and the Cr component transform skip flag are false, it is possible that the secondary transform control information is coded. That is, the LFNST identifier (lfnst_idx) is signaled.

[0050] Also, for example, when the tree type of image decoding is dual tree chroma, if at least one of the Cb component transform skip flag and the Cr component transform skip flag is true, decoding related to the secondary transform control information is omitted. Then, it is estimated that the value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, it is estimated that the value of the LFNST identifier is "0" (Infer to 0). On the other hand, if the Cb component transform skip flag and the Cr component transform skip flag are false, the bit stream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0051] An example of a syntax table related to LFNST identifiers when Method 1 is applied to the VVC described in Non-Patent Document 1 is shown in Figure 4. In this case, the encoder (or decoder) sets the variable nonTsUsed for valid components for each tree type as shown in the top row of Figure 4. This variable nonTsUsed is a variable that indicates whether or not transform skip has not been applied. If this value is true (e.g., "1"), it indicates that transform skip has not been applied. If this value is false (e.g., "0"), it indicates that transform skip has been applied.

[0052] For example, if the tree type is dual tree luma, and the Y component transform skip flag is false (treeType == DUAL_TREE_LUMA && transform_skip_flag[x0][y0][0] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). Conversely, if the Y component transform skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0053] Also, when the tree type is dual tree chroma, if the Cb component transformation skip flag and the Cr component transformation skip flag are false (treeType == DUAL_TREE_CHROMA && transform_skip_flag[x0][y0][1] == 0 && transform_skip_flag[x0][y0][2] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). On the other hand, if the Cb component transformation skip flag or the Cr component transformation skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0054] Furthermore, when the tree type is a single tree, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false (treeType == SINGLE_TREE && transform_skip_flag[x0][y0][0] == 0 && transform_skip_flag[x0][y0][1] == 0 && transform_skip_flag[x0][y0][2] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). On the other hand, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0055] Then, the variable nonTsUsed is used as one of the conditions to control whether or not to signal the LFNST identifier (lfnst_idx). For example, when the variable nonTsUsed is true (nonTsUsed == 1) and other conditions are satisfied, the encoder encodes the LFNST identifier (lfnst_idx). The decoder also performs decoding related to the LFNST identifier (lfnst_idx).

[0056] If the variable nonTsUsed is false (nonTsUsed == 0), the encoder skips encoding the LFNST identifier (lfnst_idx). The decoder skips decoding the LFNST identifier (lfnst_idx) and infers that the value of the LFNST identifier (lfnst_idx) is "0".

[0057] Fig. 5 is a diagram showing the encoding condition (decoding condition) of the LFNST identifier, which is equivalent to the syntax table in Fig. 4. For example, the encoder sets a variable NonTsUsed (upper part of Fig. 5) for each tree type. The encoder then sets the encoding condition condition of the LFNST identifier (lower part of Fig. 5) using the variable NonTsUsed as one of the conditions. If the encoding condition condition is true, the encoder signals the LFNST identifier (lfnst_idx). If the encoding condition condition is false, the encoder skips signaling the LFNST identifier (lfnst_idx).

[0058] For example, the decoder sets a variable NonTsUsed (upper part of Figure 5) for each tree type. Then, the decoder sets a decoding condition condition (lower part of Figure 5) for the LFNST identifier using the variable NonTsUsed as one of the conditions. Then, if the decoding condition condition is true, the decoder decodes the bitstream and obtains the signaled LFNST identifier (lfnst_idx). If the decoding condition condition is false, the decoder skips decoding for the LFNST identifier (lfnst_idx) and infers that the value of that LFNST identifier (lfnst_idx) is "0".

[0059] In this way, in the VVC described in Non-Patent Document 1, it is possible to suppress a decrease in coding efficiency.

[0060] Note that a negative expression may be used in the conditional expression, as in the syntax table shown in Fig. 6. The syntax table shown in Fig. 6 corresponds to the syntax table shown in Fig. 4, but uses the variable TsUsed instead of the variable nonTsUsed in Fig. 4. The variable TsUsed is a variable that indicates whether or not a conversion skip has been applied, and if this value is true (e.g., "1"), it indicates that a conversion skip has been applied. If this value is false (e.g., "0"), it indicates that a conversion skip has not been applied.

[0061] For example, the variable TsUsed is set as shown in the top row of Fig. 6. That is, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is true (transform_skip_flag[x0][y0][0] == 1 || transform_skip_flag[x0][y0][1] == 1 || transform_skip_flag[x0][y0][2] == 1), the variable TsUsed is set to true (TsUsed == 1). On the other hand, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the variable TsUsed is set to false (TsUsed == 0).

[0062] Then, using the variable TsUsed as one of the conditions, whether to signal the LFNST identifier (lfnst_idx) is controlled. For example, when the variable TsUsed is false (TsUsed == 0) and other conditions are satisfied, the encoder encodes the LFNST identifier (lfnst_idx), and the decoder performs decoding related to the LFNST identifier (lfnst_idx).

[0063] Furthermore, if the variable TsUsed is true (TsUsed == 1), the encoder skips encoding of the LFNST identifier (lfnst_idx), and the decoder skips decoding of the LFNST identifier (lfnst_idx) and infers that the value of that LFNST identifier (lfnst_idx) is "0".

[0064] Fig. 7 is a diagram showing the encoding condition (decoding condition) of the LFNST identifier, which is equivalent to the syntax table in Fig. 6. For example, the encoder sets a variable TsUsed (upper part of Fig. 7). The encoder then uses the variable TsUsed as one of the conditions to set the encoding condition of the LFNST identifier (lower part of Fig. 7). If the encoding condition condition is true, the encoder signals the LFNST identifier (lfnst_idx). If the encoding condition condition is false, the encoder skips signaling of the LFNST identifier (lfnst_idx).

[0065] For example, the decoder sets a variable TsUsed (upper part of Figure 7). Then, the decoder uses the variable TsUsed as one of the conditions to set a decoding condition condition for the LFNST identifier (lower part of Figure 7). Then, if the decoding condition condition is true, the decoder decodes the bitstream and obtains the signaled LFNST identifier (lfnst_idx). If the decoding condition condition is false, the decoder skips decoding for the LFNST identifier (lfnst_idx) and infers that the value of the LFNST identifier (lfnst_idx) is "0".

[0066] In this way, in the VVC described in Non-Patent Document 1, it is possible to suppress a decrease in coding efficiency.

[0067] <Method 2> For example, if the transform skip flag of all valid components is true, the signaling of the LFNST identifier may be skipped (method 2).

[0068] For example, if the transform skip flags of all components among the valid components in image encoding are true, the encoding of the secondary transformation control information may be omitted, and if the transform skip flag of at least one component is false, the secondary transformation control information may be encoded.

[0069] For example, if the conversion skip flags of all components among the valid components in image decoding are true, decoding of the secondary conversion control information may be omitted, and the value of the secondary conversion control information may be presumed to be a predetermined value indicating skipping of secondary conversion. If the conversion skip flag of at least one component is false, the bitstream may be decoded to generate secondary conversion control information.

[0070] In the method 2, as in the method 1, the decoding of the secondary conversion control information means decoding a bit stream generated by encoding the secondary conversion control information to generate the secondary conversion control information. In addition, the predetermined value indicating skipping the secondary conversion is arbitrary.

[0071] Furthermore, the secondary transformation control information may be any control information related to the secondary transformation, and may include, for example, a secondary transformation identifier that is an identifier indicating the type of secondary transformation (and the inverse secondary transformation).

[0072] Furthermore, the secondary transform may be any coefficient transform. For example, it may be LFNST. In that case, the inverse secondary transform may be an inverse process of LFNST. In this case, the secondary transform identifier is an LFNST identifier (lfnst_idx). Furthermore, the predetermined value indicating skipping of the secondary transform may be "0" (lfnst_idx == 0).

[0073] An example of the correspondence between combinations of tree types and transform skip flags and the signaling of LFNST identifiers when Method 2 is applied is shown in Fig. 8. Fig. 8 is a table corresponding to the table in Fig. 2. The secondary transform is LFNST, and an LFNST identifier is signaled as the secondary transform information. Also, the predetermined value indicating skipping of the secondary transform is assumed to be "0" (lfnst_idx == 0).

[0074] As in the case of FIG. 2, there are three types of tree types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Also, transform_skip_flag[0] indicates the Y component transform skip flag. transform_skip_flag[1] indicates the Cb component transform skip flag. transform_skip_flag[2] indicates the Cr component transform skip flag. The "0" and "1" in the column of each transform skip flag indicate the value of that transform skip flag. In this example, the value "1" indicates true, and the value "0" indicates false. Note that the transform skip flags (transform skip flags indicated by "-") of invalid components (Y, Cb, Cr) of each tree type are not signaled.

[0075] In this example, regardless of the tree type, if the transform skip flag of at least one of the valid components is false (value "0"), the LFNST identifier is signaled (possible). Only if the transform skip flag of all of the valid components is true (value "1"), the signaling of the LFNST identifier is skipped (omitted). In that case, the decoding side (decoder) infers the value of the LFNST identifier to be "0" (i.e., a value indicating skipping of the secondary transform).

[0076] By doing this, when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of an image are true, it is possible to skip signaling of that LFNST identifier (the bottom row of the table in FIG. 8). Even if signaling of the LFNST identifier is skipped in this way, the decoder can skip the inverse secondary transform based on that LFNST identifier. In other words, the decoder can obtain an LFNST identifier whose value is not inconsistent with the signaled transform skip flag, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse secondary transform in the decoder.

[0077] That is, by applying the above-mentioned method 2, it is possible to reduce redundancy in signaling of the LFNST identifier, and to prevent a decrease in coding efficiency.

[0078] For example, when the tree type of image coding is a single tree (SINGLE_TREE), Y, Cb, and Cr are valid components. Therefore, in this case, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are true, coding of the secondary transform control information is skipped. That is, in this case, signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is false, it is possible that the secondary transform control information is coded. That is, the LFNST identifier (lfnst_idx) is signaled.

[0079] Also, for example, when the tree type of image decoding is a single tree, if the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are true, decoding related to the secondary transform control information is skipped. Then, it is estimated that the value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, it is estimated that the value of the LFNST identifier is "0" (Infer to 0). On the other hand, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is false, the bit stream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0080] In the example of FIG. 2, when the tree type of image decoding is a single tree and the Y component transform skip flag is true (transform_skip_flag[0] == 1), the signaling of the LFNST identifier (lfnst_idx) is skipped. Therefore, even if the Cb component transform skip flag or the Cr component transform skip flag is false (transform_skip_flag[1] == 0 || transform_skip_flag[2] == 0), the secondary transform (LFNST) cannot be applied to the Cb component or the Cr component. This may reduce the coding efficiency of the Cb component or the Cr component.

[0081] On the other hand, in the case of the example of Fig. 8, as described above, if at least one of the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag is false, the LFNST identifier may be signaled. That is, even if the tree type of the image decoding is a single tree and the Y component transform skip flag is true (transform_skip_flag[0] == 1), if the Cb component transform skip flag or the Cr component transform skip flag is false (transform_skip_flag[1] == 0 || transform_skip_flag[2] == 0), the secondary transform (LFNST) can be applied to the Cb component and the Cr component. Therefore, it is possible to suppress a decrease in the coding efficiency of the Cb component and the Cr component.

[0082] For example, when the tree type of image coding is dual tree luma (DUAL_TREE_LUMA), luminance Y is a valid component. Therefore, in this case, if the Y component transform skip flag is true, the coding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if the Y component transform skip flag is false, it is possible that the secondary transform control information is coded. That is, the LFNST identifier (lfnst_idx) is signaled.

[0083] Also, for example, when the tree type of image decoding is dual tree luma, if the Y component transform skip flag is true, decoding related to the secondary transform control information is skipped. Then, the value of the secondary transform control information is estimated to be a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if the Y component transform skip flag is false, the bitstream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0084] For example, when the tree type of image coding is dual tree chroma (DUAL_TREE_CHROMA), chrominance Cb and Cr are valid components. Therefore, in this case, if the Cb component transform skip flag and the Cr component transform skip flag are true, coding of the secondary transform control information is skipped. That is, in this case, signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated to be "0" (Infer to 0). On the other hand, if at least one of the Cb component transform skip flag and the Cr component transform skip flag is false, it is possible that the secondary transform control information is coded. That is, the LFNST identifier (lfnst_idx) is signaled.

[0085] Also, for example, when the tree type of image decoding is dual tree chroma, if the Cb component transform skip flag and the Cr component transform skip flag are true, decoding related to the secondary transform control information is omitted. Then, it is estimated that the value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform. That is, in this case, decoding related to the LFNST identifier (lfnst_idx) is skipped. Then, it is estimated that the value of the LFNST identifier is "0" (Infer to 0). On the other hand, if at least one of the Cb component transform skip flag and the Cr component transform skip flag is false, the bitstream is decoded and the secondary transform control information is generated. That is, the signaled LFNST identifier (lfnst_idx) is obtained.

[0086] <Method 3> For example, when the tree type is single tree or dual tree chroma, the signaling of the LFNST identifier may be controlled in a manner similar to the method described in Non-Patent Document 1, and only when the tree type is dual tree chroma (DUAL_TREE_CHROMA), if the transform skip flags of all components among those valid in that dual tree chroma are true, the signaling of the LFNST identifier may be skipped (Method 3).

[0087] For example, if the tree type of the image encoding is dual tree chroma, and the conversion skip flags of all components among the components valid in the dual tree chroma are true, the encoding of the secondary conversion control information may be skipped, and if the conversion skip flag of at least one component is false, the secondary conversion control information may be encoded.

[0088] For example, if the tree type of image decoding is dual tree chroma, and the conversion skip flags of all components among the components valid in the dual tree chroma are true, decoding of the secondary conversion control information may be skipped, and the value of the secondary conversion control information may be presumed to be a predetermined value indicating skipping of secondary conversion. If the conversion skip flag of at least one component is false, the bitstream may be decoded to generate secondary conversion control information.

[0089] In the method 3, as in the method 1, the decoding of the secondary conversion control information means decoding a bit stream generated by encoding the secondary conversion control information to generate the secondary conversion control information. In addition, the predetermined value indicating skipping the secondary conversion is arbitrary.

[0090] Furthermore, the secondary transformation control information may be any control information related to the secondary transformation, and may include, for example, a secondary transformation identifier that is an identifier indicating the type of secondary transformation (and the inverse secondary transformation).

[0091] Furthermore, the secondary transform may be any coefficient transform. For example, it may be LFNST. In that case, the inverse secondary transform may be an inverse process of LFNST. In this case, the secondary transform identifier is an LFNST identifier (lfnst_idx). Furthermore, the predetermined value indicating skipping of the secondary transform may be "0" (lfnst_idx == 0).

[0092] An example of the correspondence between combinations of tree types and transform skip flags and the signaling of LFNST identifiers when Method 3 is applied is shown in Fig. 9. Fig. 9 is a table corresponding to the table in Fig. 2. The secondary transform is LFNST, and an LFNST identifier is signaled as the secondary transform information. Also, the predetermined value indicating skipping of the secondary transform is assumed to be "0" (lfnst_idx == 0).

[0093] As in the case of FIG. 2, there are three types of tree types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Also, transform_skip_flag[0] indicates the Y component transform skip flag. transform_skip_flag[1] indicates the Cb component transform skip flag. transform_skip_flag[2] indicates the Cr component transform skip flag. The "0" and "1" in the column of each transform skip flag indicate the value of that transform skip flag. In this example, the value "1" indicates true, and the value "0" indicates false. Note that the transform skip flags (transform skip flags indicated by "-") of invalid components (Y, Cb, Cr) of each tree type are not signaled.

[0094] In this example, if the tree type is dual tree chroma and the transform skip flag of at least one of the components valid in the dual tree chroma is false, the LFNST identifier is signaled (possible). On the other hand, if the transform skip flags of all components are true (value "1"), the coding of the LFNST identifier is skipped (omitted). In that case, the decoding side (decoder) infers the value of the LFNST identifier to be "0" (i.e., a value indicating skipping of the secondary transform).

[0095] When the tree type is single tree or dual tree lumers, the signaling of the LFNST identifier is controlled in the same manner as the method described in Non-Patent Document 1. In other words, the signaling of the LFNST identifier is controlled in the same manner as in the table of Fig. 2.

[0096] By doing this, when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, it is possible to skip signaling of the LFNST identifier (the bottom row of the table in FIG. 9). Even if signaling of the LFNST identifier is skipped in this way, the decoder can skip the inverse secondary transform based on the LFNST identifier. In other words, the decoder can obtain an LFNST identifier whose value is not inconsistent with the signaled transform skip flag, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse secondary transform in the decoder.

[0097] That is, by applying the above-mentioned method 3, it is possible to reduce redundancy in signaling of the LFNST identifier, and to prevent a decrease in coding efficiency.

[0098] <Method 4> For example, when the tree type is single tree or dual tree chroma, the signaling of the LFNST identifier may be controlled in a manner similar to the method described in Non-Patent Document 1, and only when the tree type is dual tree chroma (DUAL_TREE_CHROMA), if the transform skip flag of at least one component among the components valid in that dual tree chroma is true, the signaling of the LFNST identifier may be skipped (Method 4).

[0099] For example, if the tree type of the image encoding is dual tree chroma, and the conversion skip flag of at least one component among the components valid in the dual tree chroma is true, the encoding of the secondary conversion control information may be skipped, and if the conversion skip flags of all components are false, the secondary conversion control information may be encoded.

[0100] For example, if the tree type of image decoding is dual tree chroma and the conversion skip flag of at least one of the components valid in the dual tree chroma is true, decoding of the secondary conversion control information is skipped, and the value of the secondary conversion control information is presumed to be a predetermined value indicating skipping of the secondary conversion, and if the conversion skip flags of all components are false, the bitstream is decoded to generate secondary conversion control information.

[0101] In method 4, as in method 1, the decoding of the secondary conversion control information means decoding a bit stream generated by encoding the secondary conversion control information to generate the secondary conversion control information. In addition, the predetermined value indicating skipping the secondary conversion is arbitrary.

[0102] Furthermore, the secondary transformation control information may be any control information related to the secondary transformation, and may include, for example, a secondary transformation identifier that is an identifier indicating the type of secondary transformation (and the inverse secondary transformation).

[0103] Furthermore, the secondary transform may be any coefficient transform. For example, it may be LFNST. In that case, the inverse secondary transform may be an inverse process of LFNST. In this case, the secondary transform identifier is an LFNST identifier (lfnst_idx). Furthermore, the predetermined value indicating skipping of the secondary transform may be "0" (lfnst_idx == 0).

[0104] An example of the correspondence between combinations of tree types and transform skip flags and the signaling of LFNST identifiers when Method 4 is applied is shown in Fig. 10. Fig. 10 is a table corresponding to the table in Fig. 2. The secondary transform is LFNST, and an LFNST identifier is signaled as the secondary transform information. Also, the predetermined value indicating skipping of the secondary transform is assumed to be "0" (lfnst_idx == 0).

[0105] As in the case of FIG. 2, there are three types of tree types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Also, transform_skip_flag[0] indicates the Y component transform skip flag. transform_skip_flag[1] indicates the Cb component transform skip flag. transform_skip_flag[2] indicates the Cr component transform skip flag. The "0" and "1" in the column of each transform skip flag indicate the value of that transform skip flag. In this example, the value "1" indicates true, and the value "0" indicates false. Note that the transform skip flags (transform skip flags indicated by "-") of invalid components (Y, Cb, Cr) of each tree type are not signaled.

[0106] In this example, if the tree type is dual tree chroma and the transform skip flags of all components valid in the dual tree chroma are false, the LFNST identifier is signaled (possible). On the other hand, if the transform skip flag of at least one component is true (value "1"), the coding of the LFNST identifier is skipped (omitted). In that case, the decoding side (decoder) infers the value of the LFNST identifier to be "0" (i.e., a value indicating skipping of the secondary transform).

[0107] As with Method 3, when the tree type is single tree or dual tree lumers, the signaling of the LFNST identifier is controlled in the same manner as the method described in Non-Patent Document 1. In other words, the signaling of the LFNST identifier is controlled in the same manner as in the table of Fig. 2.

[0108] By doing this, when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, it is possible to skip signaling of the LFNST identifier (the bottom row of the table in FIG. 10). Even if signaling of the LFNST identifier is skipped in this way, the decoder can skip the inverse secondary transform based on the LFNST identifier. In other words, the decoder can obtain an LFNST identifier whose value is not inconsistent with the signaled transform skip flag, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse secondary transform in the decoder.

[0109] That is, by applying the above-mentioned method 4, it is possible to reduce redundancy in signaling of LFNST identifiers and to prevent a decrease in coding efficiency.

[0110] <Combination> The above-mentioned methods 1 to 4 can be applied in appropriate combination of part or all of them.

[0111] <2. First embodiment> <Encoding device> The present technology (Method 1 to Method 4) described in <1. Signaling Pattern of LFNST Identifier> can be applied to any device. Application examples of the present technology are described below. Fig. 11 is a block diagram showing an example of the configuration of an encoding device, which is one aspect of an image processing device to which the present technology is applied. The encoding device 100 shown in Fig. 11 is a device that encodes an LFNST identifier (lfnst_idx). The encoding device 100 performs encoding by applying, for example, CABAC (Context-based Adaptive Binary Arithmetic Code).

[0112] Note that Fig. 11 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 11. In other words, in encoding device 100, there may be processing units that are not shown as blocks in Fig. 11, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 11.

[0113] As shown in FIG. 11, an encoding device 100 includes an LFNST identifier encoding unit 101.

[0114] The LFNST identifier encoder 101 performs processing related to encoding of the LFNST identifier. For example, the LFNST identifier encoder 101 acquires an LFNST identifier (lfnst_idx) from outside. The LFNST identifier encoder 101 encodes the acquired LFNST identifier to generate a bitstream. The LFNST identifier encoder 101 outputs the generated bitstream to an external device outside the encoding device 100. This bitstream is, for example, transmitted to the decoding side (decoder) via a communication medium or the like, or stored in a storage medium.

[0115] Furthermore, the LFNST identifier encoder 101 acquires various parameters used for encoding the LFNST identifier. For example, the LFNST identifier encoder 101 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionSplitType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are described in the above-mentioned non-patent documents (for example, Non-Patent Document 1).

[0116] The LFNST identifier encoder 101 then uses these parameters to control the encoding (signaling) of the LFNST identifier (lfnst_idx), as explained in <1. LFNST identifier signaling pattern>.

[0117] For example, the LFNST identifier encoder 101 controls the encoding of this LFNST identifier by applying the above-mentioned method 1. That is, if the transform skip flag of at least one component among the valid components for image encoding is true, the LFNST identifier encoder 101 omits encoding of secondary transform control information, which is control information related to secondary transform, and if the transform skip flags of all components are false, it encodes the secondary transform control information.

[0118] For example, the LFNST identifier encoder 101 controls the encoding of the LFNST identifier in accordance with the syntax table shown in Fig. 4. Alternatively, the LFNST identifier encoder 101 may control the encoding of the LFNST identifier in accordance with the syntax table shown in Fig. 6. By following these syntax tables, the LFNST identifier encoder 101 can control the encoding of the LFNST identifier according to the tree type and conversion skip flag, as in the table shown in Fig. 3. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0119] The LFNST identifier encoder 101 may control the encoding of this LFNST identifier by applying the above-mentioned method 2. That is, the LFNST identifier encoder 101 may omit encoding of the secondary conversion control information if the transform skip flags of all components among those valid for image encoding are true, and may encode the secondary conversion control information if the transform skip flag of at least one component is false.

[0120] In this way, the LFNST identifier encoder 101 can control the encoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 8. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0121] Furthermore, the LFNST identifier encoder 101 may control the encoding of this LFNST identifier by applying the above-mentioned method 3. That is, if the tree type for image encoding is dual tree chroma and the transform skip flags of all components among those valid in that dual tree chroma are true, the LFNST identifier encoder 101 may omit encoding the secondary transform control information, and may encode the secondary transform control information if the transform skip flag of at least one component is false.

[0122] In this way, the LFNST identifier encoder 101 can control the encoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 9. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0123] Furthermore, the LFNST identifier encoder 101 may control the encoding of this LFNST identifier by applying the above-mentioned method 4. That is, if the tree type for image encoding is dual tree chroma and the transform skip flag of at least one component among the components valid in that dual tree chroma is true, the LFNST identifier encoder 101 may omit encoding the secondary transform control information, and may encode the secondary transform control information if the transform skip flags of all components are false.

[0124] In this way, the LFNST identifier encoder 101 can control the encoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 10. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0125] Regardless of which of Methods 1 to 4 is applied, the LFNST identifier encoding unit 101 can skip signaling of the LFNST identifier if the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true. Therefore, the encoding device 100 can suppress a decrease in encoding efficiency.

[0126] <Encoding process flow> Next, an example of the flow of the encoding process executed by the encoding device 100 will be described with reference to the flowchart of FIG.

[0127] When the encoding process starts, the LFNST identifier encoder 101 of the encoding device 100 derives the encoding condition "condition" of the LFNST identifier in step S101. For example, when applying method 1 described above in <1. Signaling pattern of LFNST identifier>, the LFNST identifier encoder 101 omits encoding of secondary conversion control information, which is control information related to secondary conversion, if the transform skip flag of at least one component among the valid components in image encoding is true, and encodes the secondary conversion control information if the transform skip flags of all components are false. For example, the LFNST identifier encoder 101 derives the variable NonTsUsed shown in Fig. 5, and derives the encoding condition "condition" shown in Fig. 5 using the derived variable NonTsUsed.

[0128] Alternatively, the LFNST identifier encoding unit 101 may derive the variable TsUsed shown in Fig. 7 and use the derived variable TsUsed to derive the encoding condition "condition" shown in Fig. 7. Alternatively, the LFNST identifier encoding unit 101 may apply any of methods 2 to 4 to derive the encoding condition "condition."

[0129] In step S102, the LFNST identifier encoding unit 101 determines whether the encoding condition "condition" is satisfied. In other words, the LFNST identifier encoding unit 101 determines whether the encoding condition "condition" derived in step S101 is true. If it is determined that the encoding condition "condition" is true, the process proceeds to step S103.

[0130] In step S103, the LFNST identifier encoder 101 encodes the LFNST identifier (lfnst_idx) to generate a bitstream. That is, if the encoding condition "condition" is satisfied, the LFNST identifier encoder 101 encodes the LFNST identifier (lfnst_idx) to generate a bitstream. This bitstream is supplied to a decoder via a communications medium, for example. This bitstream may also be stored in a storage medium. The encoding process ends when the process of step S103 ends.

[0131] Also, if it is determined in step S102 that the encoding condition derived in step S101 is false, the process proceeds to step S104.

[0132] In step S104, the LFNST identifier encoder 101 skips (omits) the encoding of the LFNST identifier (lfnst_idx). That is, if the encoding condition "condition" is not satisfied, the LFNST identifier encoder 101 skips the encoding of the LFNST identifier (lfnst_idx). In other words, in this case, a bitstream for the LFNST identifier is not generated (i.e., the LFNST identifier is not signaled). The encoding process ends when the process of step S104 ends.

[0133] By performing the encoding process in this manner, the LFNST identifier encoding unit 101 can skip signaling of the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true, regardless of which method is applied among methods 1 to 4. Therefore, the encoding device 100 can suppress a decrease in encoding efficiency.

[0134] <Supplementary Information> Note that the encoding device 100 (LFNST identifier encoding unit 101) is not limited to the example of an LFNST identifier, and may encode any secondary transform identifier (any secondary transform control information). In other words, the secondary transform is not limited to LFNST, and may be any coefficient transform. Also, the above-mentioned methods 1 to 4 can be applied in appropriate combinations of some or all of them.

[0135] <3. Second embodiment> <Decryption device> Fig. 13 is a block diagram showing an example of the configuration of a decoding device, which is one aspect of an image processing device to which the present technology is applied. The decoding device 200 shown in Fig. 13 is a device that decodes a bit stream (encoded data) of an LFNST identifier (lfnst_idx). The decoding device 200 performs decoding by applying a decoding method (e.g., CABAC) corresponding to the encoding method of the encoding device 100. For example, the decoding device 200 decodes a bit stream generated by the encoding device 100.

[0136] Note that Fig. 13 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 13. In other words, in the decoding device 200, there may be processing units that are not shown as blocks in Fig. 13, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 13.

[0137] As shown in FIG. 13, a decoding device 200 includes an LFNST identifier decoding unit 201.

[0138] The LFNST identifier decoder 201 performs processing related to decoding of the LFNST identifier. For example, the LFNST identifier decoder 201 acquires a bitstream from an external source. The LFNST identifier decoder 201 decodes the acquired bitstream and generates an LFNST identifier (lfnst_idx). The LFNST identifier decoder 201 outputs the generated LFNST identifier to a device external to the decoding device 200. This LFNST identifier is, for example, transmitted to the decoding side (decoder) via a communication medium or the like, or stored in a storage medium.

[0139] Furthermore, the LFNST identifier decoder 201 acquires various parameters used for decoding the LFNST identifier. For example, the LFNST identifier decoder 201 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionSplitType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are described in the above-mentioned non-patent documents (for example, Non-Patent Document 1).

[0140] These parameters are coded in an encoder (e.g., coding device 100) and supplied (i.e., signaled) as a bitstream. Decoding device 200 obtains this bitstream and decodes it to generate these parameters. Note that this decoding may be performed in a decoding unit (not shown) external to LFNST identifier decoding unit 201, or may be performed by LFNST identifier decoding unit 201.

[0141] The LFNST identifier decoder 201 then uses these parameters to control the decoding of the LFNST identifier (lfnst_idx), as explained in <1. LFNST identifier signaling pattern>.

[0142] For example, the LFNST identifier decoder 201 controls decoding related to this LFNST identifier by applying the above-mentioned method 1. That is, if the transform skip flag of at least one component among the valid components in the image decoding is true, the LFNST identifier decoder 201 omits decoding of the secondary transform control information, which is control information related to the secondary transform, and estimates that the value of that secondary transform control information is a predetermined value indicating skipping of the secondary transform, and if the transform skip flags of all components are false, it decodes the bitstream and generates secondary transform control information.

[0143] For example, the LFNST identifier decoder 201 controls the decoding of the LFNST identifier in accordance with the syntax table shown in Fig. 4. Alternatively, the LFNST identifier decoder 201 may control the decoding of the LFNST identifier in accordance with the syntax table shown in Fig. 6. By following these syntax tables, the LFNST identifier decoder 201 can control the decoding of the LFNST identifier according to the tree type and conversion skip flag, as in the table shown in Fig. 3. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0144] The LFNST identifier decoding unit 201 may control the decoding related to this LFNST identifier by applying the above-mentioned method 2. That is, if the transform skip flags of all components among the components valid for image decoding are true, the LFNST identifier decoding unit 201 may omit decoding related to the secondary conversion control information and infer that the value of the secondary conversion control information is "0." Also, if the transform skip flag of at least one component among the components valid for image decoding is false, the LFNST identifier decoding unit 201 may perform decoding related to the secondary conversion control information.

[0145] In this way, the LFNST identifier decoder 201 can control decoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 8. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0146] Furthermore, the LFNST identifier decoding unit 201 may control the decoding related to this LFNST identifier by applying the above-mentioned method 3. That is, if the tree type for image decoding is dual tree chroma and the transform skip flags of all components valid in that dual tree chroma are true, the LFNST identifier decoding unit 201 may skip (omit) the decoding related to the secondary transform control information and infer that the value of the secondary transform control information is "0." Furthermore, if the transform skip flag of at least one component valid in the image decoding is false, the LFNST identifier decoding unit 201 may perform decoding related to the secondary transform control information.

[0147] In this way, the LFNST identifier decoder 201 can control decoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 9. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0148] Furthermore, the LFNST identifier decoding unit 201 may control the decoding related to this LFNST identifier by applying the above-mentioned method 4. That is, if the tree type for image decoding is dual tree chroma and the transform skip flag of at least one component among the components valid in that dual tree chroma is true, the LFNST identifier decoding unit 201 may omit decoding related to the secondary transform control information and infer that the value of that secondary transform control information is "0." Furthermore, if the transform skip flags of all components are false, decoding related to the secondary transform control information may be performed.

[0149] In this way, the LFNST identifier decoder 201 can control decoding of the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 10. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0150] Regardless of which of Methods 1 to 4 is applied, the LFNST identifier decoding unit 201 can skip decoding of the LFNST identifier if the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb component and the Cr component of the image are true. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0151] <Decryption process flow> Next, an example of the flow of the decoding process executed by the decoding device 200 will be described with reference to the flowchart of FIG.

[0152] When the decoding process starts, in step S201, the LFNST identifier decoding unit 201 of the decoding device 200 derives the decoding condition "condition" of the LFNST identifier. For example, when applying method 1 described above in <1. Signaling pattern of LFNST identifier>, the LFNST identifier decoding unit 201 omits decoding of secondary conversion control information, which is control information related to secondary conversion, if the conversion skip flag of at least one component among the valid components in the image decoding is true, and performs decoding of the secondary conversion control information if the conversion skip flags of all components are false. For example, the LFNST identifier decoding unit 201 derives the variable NonTsUsed shown in Fig. 5, and derives the decoding condition "condition" shown in Fig. 5 using the derived variable NonTsUsed.

[0153] Alternatively, the LFNST identifier decoding unit 201 may derive the variable TsUsed shown in Fig. 7 and use the derived variable TsUsed to derive the decoding condition condition shown in Fig. 7. Alternatively, the LFNST identifier decoding unit 201 may apply any of methods 2 to 4 to derive the decoding condition condition.

[0154] In step S202, the LFNST identifier decoding unit 201 determines whether the decoding condition "condition" is satisfied. In other words, the LFNST identifier decoding unit 201 determines whether the decoding condition "condition" derived in step S201 is true. If it is determined that the decoding condition "condition" is true, the process proceeds to step S203.

[0155] In step S203, the LFNST identifier decoder 201 decodes the bitstream and generates an LFNST identifier (lfnst_idx). That is, if the decoding condition "condition" is satisfied, the LFNST identifier decoder 201 decodes the bitstream and generates an LFNST identifier (lfnst_idx). When the process of step S203 ends, the decoding process ends.

[0156] Also, if it is determined in step S202 that the decoding condition "condition" derived in step S201 is false, the process proceeds to step S204.

[0157] In step S204, the LFNST identifier decoder 201 skips (omits) the decoding of the LFNST identifier (lfnst_idx). In other words, if the decoding condition "condition" is not satisfied, the LFNST identifier decoder 201 skips the decoding of the LFNST identifier (lfnst_idx) and estimates that the value of the LFNST identifier is "0" (lfnst_idx = 0). The decoding process ends when the process of step S204 ends.

[0158] By performing the decoding process in this manner, the LFNST identifier decoding unit 201 can skip decoding of the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb component and the Cr component of the image are true, regardless of which method is applied among methods 1 to 4. Therefore, the decoding device 200 can suppress a decrease in encoding efficiency.

[0159] <Supplementary Information> Note that decoding device 200 (LFNST identifier decoding unit 201) may perform decoding of any secondary transform identifier (any secondary transform control information) without being limited to the example of an LFNST identifier. In other words, the inverse secondary transform is not limited to the inverse processing of LFNST, but may be any inverse coefficient transform. Also, the above-mentioned methods 1 to 4 may be applied in appropriate combinations of some or all of them.

[0160] <4. Third embodiment> <Image encoding device> Fig. 15 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 300 shown in Fig. 15 is a device that encodes image data of a moving image. For example, the image encoding device 300 can encode image data of a moving image using an encoding method described in any of the above non-patent documents.

[0161] Note that Fig. 15 shows the main processing units (blocks) and data flows, etc., and is not limited to all that is shown in Fig. 15. In other words, in the image encoding device 300, there may be processing units that are not shown as blocks in Fig. 15, and there may be processes and data flows that are not shown as arrows or the like in Fig. 15.

[0162] As shown in FIG. 15, the image encoding device 300 has a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transformation unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transformation unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0163] <Control Unit> The control unit 301 divides the video data held by the rearrangement buffer 311 into blocks (CU, PU, ​​transform block, etc.) of processing units based on an external or pre-specified block size of processing units. The control unit 301 also determines coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0164] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 301 supplies them to each block. More specifically, the process is as follows.

[0165] Header information Hinfo is provided for each block. The prediction mode information Pinfo is supplied to the encoding unit 315 and the prediction unit 322 . The transformation information Tinfo is supplied to the encoding unit 315 , the orthogonal transformation unit 313 , the quantization unit 314 , the inverse quantization unit 317 , and the inverse orthogonal transformation unit 318 . The filter information Finfo is supplied to the in-loop filter unit 320 .

[0166] <Sorting Buffer> To the image coding device 300, each field (input image) of video image data is input in its playback order (display order). The reordering buffer 311 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 301, the reordering buffer 311 reorders the input images in coding order (decoding order) and divides them into blocks of processing units. The reordering buffer 311 supplies each processed input image to the calculation unit 312. The reordering buffer 311 also supplies each input image (original image) to the prediction unit 322 and the in-loop filter unit 320.

[0167] <Arithmetic section> The calculation unit 312 receives as input an image I corresponding to a block of a processing unit and a predicted image P supplied from the prediction unit 322, subtracts the predicted image P from the image I as shown in the following equation, derives a prediction residual D, and supplies it to the orthogonal transformation unit 313.

[0168] D = I - P

[0169] <Orthogonal transformation section> The orthogonal transform unit 313 performs processing related to coefficient transform. For example, the orthogonal transform unit 313 obtains the prediction residual D supplied from the calculation unit 312. In addition, the orthogonal transform unit 313 obtains the transform information Tinfo supplied from the control unit 301.

[0170] The orthogonal transform unit 313 performs an orthogonal transform on the prediction residual D based on the transform information Tinfo to derive a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient, and performs a secondary transform on the primary transform coefficient to generate a secondary transform coefficient. The orthogonal transform unit 313 supplies the obtained secondary transform coefficient to the quantization unit 314 as the transform coefficient Coeff.

[0171] Note that the orthogonal transform is an example of coefficient transform, and is not limited to this example. That is, the orthogonal transform unit 313 can perform any coefficient transform on the prediction residual D. In addition, the orthogonal transform unit 313 can perform any coefficient transform as the primary transform and the secondary transform.

[0172] <Quantization part> The quantization unit 314 performs processing related to quantization. For example, the quantization unit 314 acquires the transform coefficient Coeff supplied from the orthogonal transformation unit 313. The quantization unit 314 also acquires the transform information Tinfo supplied from the control unit 301. The quantization unit 314 then scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. Any method of quantization may be used. The rate of quantization is controlled by the rate control unit 323. The quantization unit 314 supplies the quantized transform coefficient obtained by such quantization, i.e., the quantized transform coefficient level, to the encoding unit 315 and the inverse quantization unit 317.

[0173] <Encoding part> The encoding unit 315 performs processing related to encoding. For example, the encoding unit 315 acquires a quantized transform coefficient level level supplied from the quantization unit 314. The encoding unit 315 also acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) supplied from the control unit 301. The encoding unit 315 also acquires information related to filters such as filter coefficients supplied from the in-loop filter unit 320. The encoding unit 315 also acquires information related to an optimal prediction mode supplied from the prediction unit 322.

[0174] The encoding unit 315 performs variable length coding (for example, arithmetic coding) on ​​the quantized transform coefficient level level to generate a bit string (coded data). The encoding unit 315 also derives residual information Rinfo from the quantized transform coefficient level level, codes the residual information Rinfo, and generates a bit string.

[0175] The encoding unit 315 includes, in the filter information Finfo, information on the filter supplied from the in-loop filter unit 320. The encoding unit 315 also includes, in the prediction mode information Pinfo, information on the optimal prediction mode supplied from the prediction unit 322. Then, the encoding unit 315 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0176] The encoding unit 315 multiplexes the bit strings of the various information generated as described above to generate encoded data. The encoding unit 315 supplies the encoded data to the accumulation buffer 316.

[0177] <Accumulation buffer> The accumulation buffer 316 temporarily holds the encoded data obtained in the encoding unit 315. The accumulation buffer 316 outputs the held encoded data, for example as a bit stream, to the outside of the image encoding device 300 at a predetermined timing. For example, this encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, etc. In other words, the accumulation buffer 316 is also a transmission unit that transmits the encoded data (bit stream).

[0178] <Inverse quantization section> The inverse quantization unit 317 performs processing related to inverse quantization. For example, the inverse quantization unit 317 obtains the quantized transformation coefficient level level supplied from the quantization unit 314. In addition, the inverse quantization unit 317 obtains the transformation information Tinfo supplied from the control unit 301.

[0179] The inverse quantization unit 317 scales (inverse quantizes) the value of the quantized transform coefficient level level based on the transform information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed by the quantization unit 314. The inverse quantization unit 317 supplies the transform coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.

[0180] <Inverse orthogonal transform section> The inverse orthogonal transform unit 318 performs processing related to inverse coefficient transform. For example, the inverse orthogonal transform unit 318 obtains the transform coefficient Coeff_IQ supplied from the inverse quantization unit 317. The inverse orthogonal transform unit 318 also obtains the transform information Tinfo supplied from the control unit 301.

[0181] The inverse orthogonal transform unit 318 performs an inverse orthogonal transform on the transform coefficients Coeff_IQ based on the transform information Tinfo, and derives a prediction residual D'. Note that this inverse orthogonal transform is an inverse process of the orthogonal transform performed in the orthogonal transform unit 313. For example, the inverse orthogonal transform unit 318 performs an inverse secondary transform on the transform coefficients Coeff_IQ (secondary transform coefficients) to generate primary transform coefficients. Also, the inverse orthogonal transform unit 318 performs an inverse primary transform on the primary transform coefficients to generate a prediction residual D'. Note that the inverse secondary transform is an inverse process of the secondary transform performed in the orthogonal transform unit 313. Also, the inverse primary transform is an inverse process of the primary transform performed in the orthogonal transform unit 313.

[0182] The inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 319. Note that since the inverse orthogonal transform unit 318 is similar to an inverse orthogonal transform unit (described later) on the decoding side, the explanation on the decoding side (described later) can be applied to the inverse orthogonal transform unit 318.

[0183] <Arithmetic section> The calculation unit 319 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 318 and the predicted image P supplied from the prediction unit 322. The calculation unit 319 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to derive a local decoded image Rlocal. The calculation unit 319 supplies the derived local decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

[0184] <In-loop filter section> The in-loop filter unit 320 performs processing related to in-loop filter processing. For example, the in-loop filter unit 320 receives as input the locally decoded image Rlocal supplied from the calculation unit 319, the filter information Finfo supplied from the control unit 301, and the input image (original image) supplied from the rearrangement buffer 311. Note that any information is input to the in-loop filter unit 320, and information other than these pieces of information may be input. For example, a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, information on a block (CU, CTU, etc.), etc. may be input to the in-loop filter unit 320 as necessary.

[0185] The in-loop filter unit 320 performs an appropriate filter process on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses an input image (original image) and other input information for the filter process as necessary.

[0186] For example, the in-loop filter unit 320 applies four in-loop filters, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Note that which filters are applied and in what order they are applied are arbitrary and can be appropriately selected.

[0187] Of course, the filtering process performed by the in-loop filter unit 320 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 320 may apply a Wiener filter or the like.

[0188] The in-loop filter unit 320 supplies the filtered local decoded image Rlocal to the frame memory 321. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 320 supplies the information about the filter to the encoding unit 315.

[0189] <Frame memory> The frame memory 321 performs processing related to storage of image-related data. For example, the frame memory 321 receives the locally decoded image Rlocal supplied from the calculation unit 319 and the locally decoded image Rlocal that has been subjected to filtering processing supplied from the in-loop filter unit 320, and holds (stores) them. The frame memory 321 also reconstructs a decoded image R for each picture unit using the locally decoded image Rlocal, and holds it (stores it in a buffer in the frame memory 321). The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.

[0190] <Prediction Department> The prediction unit 322 performs processing related to generation of a predicted image. For example, the prediction unit 322 obtains prediction mode information Pinfo supplied from the control unit 301. The prediction unit 322 also obtains an input image (original image) supplied from the sorting buffer 311. The prediction unit 322 also obtains a decoded image R (or a part thereof) to be read from the frame memory 321.

[0191] The prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and an input image (original image). That is, the prediction unit 322 performs prediction by referring to the decoded image R as a reference image, performs motion compensation processing based on the prediction result, and generates a predicted image P.

[0192] The prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319. In addition, the prediction unit 322 supplies information on the prediction mode selected by the above processing, that is, the optimal prediction mode, to the encoding unit 315 as necessary.

[0193] <Rate control section> The rate control unit 323 performs processes related to rate control. For example, based on the amount of code of the encoded data stored in the accumulation buffer 316, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 so that overflow or underflow does not occur.

[0194] <Encoding Control of LFNST Identifier> For the image encoding apparatus 300 configured as described above, the present technology described in <1. Signaling Pattern of LFNST Identifier> can be applied.

[0195] For example, the orthogonal transformation unit 313 performs LFNST as a secondary transformation and supplies an LFNST identifier (lfnst_idx) to the encoding unit 315 as secondary transformation control information (secondary transformation identifier) (arrow 331). Note that the orthogonal transformation unit 313 also supplies the LFNST identifier (lfnst_idx) to the inverse orthogonal transformation unit 318 (arrow 332).

[0196] The encoding unit 315 includes an LFNST identifier encoding unit 101 of the encoding apparatus 100 (FIG. 11). The LFNST identifier encoding unit 101 acquires the LFNST identifier supplied from the orthogonal transformation unit 313. The LFNST identifier encoding unit 101 encodes the acquired LFNST identifier and generates a bit string. The encoding unit 315 multiplexes the generated bit string with the bit strings of other information to generate encoded data. The encoding unit 315 supplies the encoded data to the accumulation buffer 316.

[0197] The LFNST identifier encoder 101 also acquires various parameters used to control the encoding of the LFNST identifier as shown in Fig. 11, which are supplied as transformation information Tinfo from the controller 301. For example, the LFNST identifier encoder 101 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionSplitType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are described in the above-mentioned non-patent documents (for example, Non-Patent Document 1).

[0198] The LFNST identifier encoder 101 (i.e., encoder 315) then uses these parameters to control the encoding (signaling) of the LFNST identifier (lfnst_idx), as explained in <1. Signaling pattern of LFNST identifier>.

[0199] For example, the encoding unit 315 controls the encoding of this LFNST identifier by applying the above-mentioned method 1. That is, if the transform skip flag of at least one component among the components that are valid for image encoding is true, the encoding unit 315 omits encoding of secondary transform control information, which is control information related to secondary transform, and if the transform skip flags of all components are false, the encoding unit 315 encodes the secondary transform control information.

[0200] For example, the encoder 315 controls the encoding of the LFNST identifier in accordance with the syntax table shown in Fig. 4. Alternatively, the encoder 315 may control the encoding of the LFNST identifier in accordance with the syntax table shown in Fig. 6. By following these syntax tables, the encoder 315 can control the encoding of the LFNST identifier depending on the tree type and conversion skip flag, as in the table shown in Fig. 3. The control method for each tree type is as described above in <1. LFNST identifier signaling pattern>.

[0201] The encoding unit 315 may control the encoding of this LFNST identifier by applying the above-mentioned method 2. That is, the encoding unit 315 may omit encoding of the secondary conversion control information if the transform skip flags of all components among the components valid for image encoding are true, and may encode the secondary conversion control information if the transform skip flag of at least one component is false.

[0202] In this way, the encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the conversion skip flag, as shown in the table in Fig. 8. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0203] Furthermore, the encoding unit 315 may control the encoding of this LFNST identifier by applying the above-mentioned method 3. That is, if the tree type of image encoding is dual tree chroma and the transform skip flags of all components among the components valid in the dual tree chroma are true, the encoding unit 315 may omit encoding of the secondary transform control information, and may encode the secondary transform control information if the transform skip flag of at least one component is false.

[0204] In this way, the encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the conversion skip flag, as shown in the table in Fig. 9. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0205] Furthermore, the encoding unit 315 may control the encoding of this LFNST identifier by applying the above-mentioned method 4. That is, if the tree type of image encoding is dual tree chroma and the transform skip flag of at least one component among the components valid in the dual tree chroma is true, the encoding unit 315 may omit encoding of the secondary transform control information, and if the transform skip flags of all components are false, the encoding unit 315 may encode the secondary transform control information.

[0206] In this way, the encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the conversion skip flag, as shown in the table in Fig. 10. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0207] In any of methods 1 to 4, the encoding unit 315 can skip signaling of the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true. Therefore, the image encoding device 300 can suppress a decrease in encoding efficiency.

[0208] <Configuration example> These processing units (the control unit 301, the rearrangement buffer 311 to the rate control unit 323) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.

[0209] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 300 configured as above will be described with reference to the flowchart of FIG.

[0210] When the image encoding process starts, in step S301, the reordering buffer 311 is controlled by the control unit 301 to reorder the frames of the input video data from the display order to the encoding order.

[0211] In step S302, the control unit 301 sets a processing unit for the input image held by the sorting buffer 311 (performs block division).

[0212] In step S303, the control unit 301 determines (sets) coding parameters for the input image held by the reordering buffer 311.

[0213] In step S304, the prediction unit 322 performs a prediction process to generate a predicted image, etc., of an optimal prediction mode. For example, in this prediction process, the prediction unit 322 performs intra prediction to generate a predicted image, etc., of an optimal intra prediction mode, performs inter prediction to generate a predicted image, etc., of an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value, etc.

[0214] In step S305, the calculation unit 312 calculates the difference between the input image and the prediction image of the optimal mode selected by the prediction process in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the prediction image. The prediction residual D thus obtained has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to the case where the image is encoded as it is.

[0215] In step S306, the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated by the process of step S305 to derive a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient, and performs a secondary transform on the primary transform coefficient to generate a secondary transform coefficient (transform coefficient Coeff).

[0216] In step S307, the quantization unit 314 quantizes the transform coefficient Coeff obtained by the process of step S306 by using the quantization parameter calculated by the control unit 301, etc., to derive the quantized transform coefficient level level.

[0217] In step S308, the inverse quantization unit 317 inverse quantizes the quantized transform coefficient level generated by the process of step S307 with characteristics corresponding to the quantization characteristics of step S307, to derive the transform coefficient Coeff_IQ.

[0218] In step S309, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S308 using a method corresponding to the orthogonal transform process of step S306 to derive a prediction residual D'. For example, the inverse orthogonal transform unit 318 performs inverse secondary transform on the transform coefficient Coeff_IQ (secondary transform coefficient) to generate a primary transform coefficient. The inverse orthogonal transform unit 318 also performs inverse primary transform on the primary transform coefficient to generate a prediction residual D'.

[0219] This inverse orthogonal transform process is the same as the inverse orthogonal transform process performed on the decoding side, and therefore the description of the decoding side to be described later can be applied to the inverse orthogonal transform process in step S309.

[0220] In step S310, the calculation unit 319 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S304 to the prediction residual D' derived in the process in step S309.

[0221] In step S311, the in-loop filter unit 320 performs an in-loop filter process on the locally decoded image derived in the process of step S310.

[0222] In step S312, the frame memory 321 stores the locally decoded image derived by the process of step S310 and the locally decoded image filtered in step S311.

[0223] In step S313, the encoding unit 315 encodes the quantized transform coefficient level level obtained by the process of step S307. For example, the encoding unit 315 encodes the quantized transform coefficient level level, which is information about the image, by arithmetic coding or the like to generate encoded data. At this time, the encoding unit 315 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo). Furthermore, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level level and encodes the residual information RInfo.

[0224] In step S314, the accumulation buffer 316 accumulates the encoded data thus obtained and outputs it to the outside of the image encoding apparatus 300 as, for example, a bit stream. This bit stream is transmitted to the decoding side via, for example, a transmission path or a recording medium. Also, the rate control unit 323 performs rate control as necessary.

[0225] When the process of step S314 ends, the image encoding process ends.

[0226] <Encoding Control of LFNST Identifier> The present technology described in <1. Signaling Pattern of LFNST Identifier> can be applied to the image encoding process with the above flow.

[0227] For example, in step S306, the orthogonal transform unit 313 performs LFNST as a secondary transform. The orthogonal transform unit 313 generates an LFNST identifier (lfnst_idx) as secondary transform control information (secondary transform identifier) for the secondary transform (LFNST).

[0228] In step S313, the encoding unit 315 (the LFNST identifier encoding unit 101) encodes the LFNST identifier. At that time, the encoding unit 315 controls the encoding of the LFNST identifier based on the conversion information Tinfo. In other words, the encoding unit 315 performs the encoding process shown in the flowchart in Fig. 12 as one process of step S313.

[0229] For example, in step S101, the encoding unit 315 derives an encoding condition for the LFNST identifier based on the various parameters included in the conversion information Tinfo described above. In step S102, the encoding unit 315 determines whether the encoding condition is satisfied. If it is determined that the encoding condition is true, in step S103, the encoding unit 315 encodes the LFNST identifier to generate a bit string. The encoding unit 315 multiplexes the generated bit string with bit strings of other information to generate encoded data. That is, in this case, the LFNST identifier is signaled.

[0230] If it is determined in step S102 that the encoding condition derived in step S101 is false, the encoding unit 315 skips (omits) encoding the LFNST identifier in step S104. In other words, in this case, a bit string for the LFNST identifier is not generated. In other words, the LFNST identifier is not signaled.

[0231] In the encoding process described above, the encoding unit 315 (the LFNST identifier encoding unit 101) applies, for example, method 1 described above in <1. LFNST identifier signaling pattern>. That is, if the transform skip flag of at least one component among the valid components in image encoding is true, the encoding unit 315 omits encoding of secondary transform control information, which is control information related to secondary transform, and if the transform skip flags of all components are false, it encodes the secondary transform control information.

[0232] In that case, in step S101, the encoding unit 315 derives the variable NonTsUsed shown in Fig. 5, and uses the derived variable NonTsUsed to derive the encoding condition condition shown in Fig. 5. Note that the encoding unit 315 may derive the variable TsUsed shown in Fig. 7, and uses the derived variable TsUsed to derive the encoding condition condition shown in Fig. 7. Also, the encoding unit 315 may apply any one of methods 2 to 4 to derive the encoding condition condition.

[0233] By performing the image coding process in this manner, the coding unit 315 can skip signaling of the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true, regardless of which method is applied among methods 1 to 4. Therefore, the image coding device 300 can suppress a decrease in coding efficiency.

[0234] <Supplementary Information> Note that the image coding device 300 can apply any coefficient transform as the secondary transform. Therefore, the coding unit 315 can code any secondary transform identifier (any secondary transform control information) without being limited to the example of the LFNST identifier. Also, the above-mentioned methods 1 to 4 can be applied in a suitable combination of some or all of them.

[0235] <5. Fourth embodiment> <Image Decoding Device> Fig. 17 is a block diagram showing an example of a configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 400 shown in Fig. 17 is a device that encodes encoded data of a moving image. For example, the image decoding device 400 can decode encoded data by a decoding method described in any of the above-mentioned non-patent documents. For example, the image decoding device 400 decodes encoded data (bit stream) generated by the above-mentioned image encoding device 300.

[0236] Note that Fig. 17 shows the main processing units (blocks) and data flows, etc., and does not necessarily show everything in Fig. 17. That is, in the image decoding device 400, there may be processing units that are not shown as blocks in Fig. 17, and there may be processes and data flows that are not shown as arrows or the like in Fig. 17.

[0237] 17, the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transformation unit 414, a calculation unit 415, an in-loop filter unit 416, a rearrangement buffer 417, a frame memory 418, and a prediction unit 419. The prediction unit 419 includes an intra prediction unit and an inter prediction unit, not shown. The image decoding device 400 is a device for generating moving image data by decoding encoded data (bit stream).

[0238] <Accumulation buffer> The accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400. The accumulation buffer 411 supplies the accumulated bitstream to the decoding unit 412 at a predetermined timing or when a predetermined condition is met.

[0239] <Decryption section> The decoding unit 412 performs processing for decoding images. For example, the decoding unit 412 obtains a bit stream supplied from the accumulation buffer 411. The decoding unit 412 also performs variable-length decoding of the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, and derives parameters.

[0240] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, and other information. That is, the decoding unit 412 parses (analyzes and obtains) these pieces of information from the bitstream. These pieces of information will be described below.

[0241] <Header information Hinfo> The header information Hinfo includes, for example, header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), and a slice header (SH). The header information Hinfo includes, for example, information that specifies an image size (horizontal width PicWidth, vertical width PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum value MaxCUSize / minimum value MinCUSize of CU size, a maximum depth MaxQTDepth / minimum depth MinQTDepth of quad-tree partitioning, a maximum depth MaxBTDepth / minimum depth MinBTDepth of binary-tree partitioning, a maximum value MaxTSSize of transform skip blocks (also called maximum transform skip block size), an on / off flag (also called valid flag) of each encoding tool, and the like.

[0242] For example, the on / off flag of the encoding tool included in the header information Hinfo includes on / off flags related to the conversion and quantization processes shown below. The on / off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax related to the encoding tool is present in the encoded data. When the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable, and when the value of the on / off flag is 0 (false), it indicates that the encoding tool is not usable. The interpretation of the flag value may be reversed.

[0243] For example, the header information Hinfo may include an inter-component prediction enable flag (ccp_enabled_flag). The inter-component prediction enable flag is flag information indicating whether inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available or not. For example, when this flag information is "1" (true), it indicates that it is available, and when it is "0" (false), it indicates that it is not available.

[0244] This CCP is also called component-to-component linear prediction (CCLM or CCLMP).

[0245] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and other information.

[0246] The intra prediction mode information IPinfo may include, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from that syntax.

[0247] Furthermore, the intra prediction mode information IPinfo may include, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)). The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether inter-component linear prediction is applied or not. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0248] Furthermore, the intra-prediction mode information IPinfo may include a multi-class linear prediction mode flag (mclm_flag). The multi-class linear prediction mode flag (mclm_flag) is information on the mode of linear prediction (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether or not to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0249] Furthermore, the intra prediction mode information IPinfo may include a chroma sample location type identifier (chroma_sample_loc_type_idx). The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of pixel location of a chroma component (also referred to as a chroma sample location type). For example, when the chroma array type (ChromaArrayType), which is information related to the color format, indicates the 420 format, the chroma sample location type identifier is assigned as shown in the following formula.

[0250] chroma_sample_loc_type_idx == 0:Type2 chroma_sample_loc_type_idx == 1:Type3 chroma_sample_loc_type_idx == 2:Type0 chroma_sample_loc_type_idx == 3:Type1

[0251] It should be noted that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted as information on the pixel location of the chrominance component (chroma_sample_loc_info()) (i.e., stored in information on the pixel location of the chrominance component).

[0252] Furthermore, the intra prediction mode information IPinfo may include a chroma MPM identifier (chroma_mpm_idx). The chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is to be specified as a chroma intra prediction mode.

[0253] Furthermore, the intra-prediction mode information IPinfo may include a luma intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0254] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0255] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0256] <Conversion information Tinfo> The conversion information Tinfo may include, for example, the width size TBWSize and the height TBHSize of the conversion block to be processed. Note that, instead of the width size TBWSize of the conversion block to be processed, a logarithmic value log2TBWSize with a base of 2 may be applied. Also, instead of the height TBHSize of the conversion block to be processed, a logarithmic value log2TBHSize with a base of 2 may be applied.

[0257] The transform information Tinfo may also include a transform skip flag (transform_skip_flag (or ts_flag)). This transform skip flag is a flag indicating whether or not to skip the coefficient transform (or the inverse coefficient transform). Note that this transform skip flag may be signaled for each of the Y, Cb, and Cr components (transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2]).

[0258] Furthermore, the conversion information Tinfo may include parameters such as a scan identifier (scanIdx), a quantization parameter (qp), and a quantization matrix (scaling_matrix (eg, JCTVC-W1005, 7.3.4 Scaling list data syntax)).

[0259] Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0260] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax of JCTVC-W1005) may include, for example, a residual data presence / absence flag (cbf (coded_block_flag)). The residual information Rinfo may also include a last non-zero coefficient X coordinate (last_sig_coeff_x_pos) and a last non-zero coefficient Y coordinate (last_sig_coeff_y_pos). The residual information Rinfo may also include a sub-block non-zero coefficient presence / absence flag (coded_sub_block_flag) and a non-zero coefficient presence / absence flag (sig_coeff_flag).

[0261] The residual information Rinfo may also include a GR1 flag (gr1_flag) which is a flag indicating whether the level of the non-zero coefficient is greater than 1, and a GR2 flag (gr2_flag) which is a flag indicating whether the level of the non-zero coefficient is greater than 2. The residual information Rinfo may also include a sign code (sign_flag) which is a code indicating the positive or negative of the non-zero coefficient. The residual information Rinfo may also include a non-zero coefficient residual level (coeff_abs_level_remaining) which is the residual level of the non-zero coefficient.

[0262] Of course, the information included in the residual information Rinfo is arbitrary, and information other than the above information may be included.

[0263] <Filter information Finfo> The filter information Finfo includes control information related to filter processing. For example, the filter information Finfo may include control information related to a deblocking filter (DBF). The filter information Finfo may also include control information related to a pixel adaptive offset (SAO). The filter information Finfo may also include control information related to an adaptive loop filter (ALF). The filter information Finfo may also include control information related to other linear and nonlinear filters.

[0264] For example, the filter information Finfo may include information specifying a picture to which each filter is applied or an area within the picture. The filter information Finfo may also include filter On / Off control information for each CU. The filter information Finfo may also include filter On / Off control information related to slice or tile boundaries.

[0265] Of course, the information included in the filter information Finfo is arbitrary, and information other than the above information may be included.

[0266] Returning to the description of the decoding unit 412, the decoding unit 412 derives the quantized transform coefficient level "level" of each coefficient position in each transform block by referring to the residual information Rinfo. The decoding unit 412 supplies the quantized transform coefficient level "level" to the inverse quantization unit 413.

[0267] Furthermore, the decoding unit 412 supplies the parsed header information Hinfo, prediction mode information Pinfo, quantization transformation coefficient level level, transformation information Tinfo, and filter information Finfo to each block.

[0268] The header information Hinfo is supplied to the inverse quantization unit 413, the inverse orthogonal transformation unit 414, the prediction unit 419, and the in-loop filter unit 416. The prediction mode information Pinfo is supplied to the inverse quantization unit 413 and the prediction unit 419. The transformation information Tinfo is supplied to the inverse quantization unit 413 and the inverse orthogonal transformation unit 414. The filter information Finfo is supplied to the in-loop filter unit 416.

[0269] Of course, the above example is merely an example, and the present invention is not limited to this example. For example, each encoding parameter may be supplied to an arbitrary processing unit. In addition, other information may be supplied to an arbitrary processing unit.

[0270] <Inverse quantization section> The inverse quantization unit 413 performs processing related to inverse quantization. For example, the inverse quantization unit 413 acquires the transformation information Tinfo and the quantized transformation coefficient level level supplied from the decoding unit 412. Furthermore, the inverse quantization unit 413 scales (inverse quantizes) the value of the quantized transformation coefficient level level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization.

[0271] This inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 314 of the image encoding device 300. This inverse quantization is similar to the inverse quantization performed by the inverse quantization unit 317 of the image encoding device 300. In other words, the inverse quantization unit 317 performs the same process (inverse quantization) as the inverse quantization unit 413.

[0272] The inverse quantization unit 413 supplies the derived transformation coefficient Coeff_IQ to the inverse orthogonal transformation unit 414 .

[0273] <Inverse orthogonal transform section> The inverse orthogonal transform unit 414 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 414 obtains the transform coefficient Coeff_IQ supplied from the inverse quantization unit 413. The inverse orthogonal transform unit 414 also obtains the transform information Tinfo supplied from the decoding unit 412.

[0274] The inverse orthogonal transform unit 414 performs an inverse orthogonal transform process on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficient Coeff_IQ to generate a primary transform coefficient, and performs a primary transform on the primary transform coefficient to generate a prediction residual D'.

[0275] This inverse orthogonal transform is performed as an inverse process of the orthogonal transform performed by the orthogonal transform unit 313 of the image encoding device 300. This inverse orthogonal transform is similar to the inverse orthogonal transform performed by the inverse orthogonal transform unit 318 of the image encoding device 300. In other words, the inverse orthogonal transform unit 318 performs the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 414.

[0276] The inverse orthogonal transform unit 414 supplies the derived prediction residual D′ to the calculation unit 415 .

[0277] <Arithmetic section> The calculation unit 415 performs processing related to addition of information related to an image. For example, the calculation unit 415 obtains a prediction residual D' supplied from the inverse orthogonal transformation unit 414. The calculation unit 415 also obtains a predicted image P supplied from the prediction unit 419. As shown in the following formula, the calculation unit 415 adds the prediction residual D' and a predicted image P (prediction signal) corresponding to the prediction residual D' to derive a locally decoded image Rlocal.

[0278] Rlocal = D' + P

[0279] The calculation unit 415 supplies the derived locally decoded image Rlocal to the in-loop filter unit 416 and the frame memory 418.

[0280] <In-loop filter section> The in-loop filter unit 416 performs processing related to in-loop filter processing. For example, the in-loop filter unit 416 acquires a local decoded image Rlocal supplied from the calculation unit 415. The in-loop filter unit 416 also acquires filter information Finfo supplied from the decoding unit 412. Note that any information is input to the in-loop filter unit 416, and information other than the above information may be input.

[0281] The in-loop filter unit 416 performs appropriate filtering on the local decoded image Rlocal based on the filter information Finfo. For example, the in-loop filter unit 416 applies four in-loop filters, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Note that which filters are applied and in what order they are applied are arbitrary and can be appropriately selected.

[0282] The in-loop filter unit 416 performs a filter process corresponding to the filter process performed by the encoding side (for example, the in-loop filter unit 320 of the image encoding device 300). Of course, the filter process performed by the in-loop filter unit 416 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 416 may apply a Wiener filter or the like.

[0283] The in-loop filter unit 416 supplies the filtered locally decoded image Rlocal to a reordering buffer 417 and a frame memory 418.

[0284] <Sorting Buffer> The reordering buffer 417 receives the locally decoded image Rlocal supplied from the in-loop filter unit 416 and holds (stores) it. The reordering buffer 417 reconstructs a decoded image R for each picture using the locally decoded image Rlocal and holds it (stores it in the buffer). The reordering buffer 417 reorders the obtained decoded images R from the decoding order to the playback order. The reordering buffer 417 outputs the reordered decoded images R to the outside of the image decoding device 400 as video data.

[0285] <Frame memory> The frame memory 418 performs processing related to storage of image-related data. For example, the frame memory 418 acquires a locally decoded image Rlocal supplied from the calculation unit 415. Then, the frame memory 418 reconstructs a decoded image R for each picture unit using the locally decoded image Rlocal. The frame memory 418 stores the reconstructed decoded image R in a buffer within the frame memory 418.

[0286] The frame memory 418 also acquires the in-loop filtered locally decoded image Rlocal supplied from the in-loop filter unit 416. The frame memory 418 then reconstructs a decoded image R for each picture using the in-loop filtered locally decoded image Rlocal. The frame memory 418 stores the reconstructed decoded image R in a buffer within the frame memory 418.

[0287] Furthermore, the frame memory 418 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 419 as a reference image.

[0288] The frame memory 418 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like, which are related to the generation of a decoded image.

[0289] <Prediction Department> The prediction unit 419 performs processing related to the generation of a predicted image. For example, the prediction unit 419 acquires prediction mode information Pinfo supplied from the decoding unit 412. Further, the prediction unit 419 performs prediction by the prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. At the time of the derivation, the prediction unit 419 uses the decoded image R (or a part thereof) stored in the frame memory 418, which is specified by the prediction mode information Pinfo, as a reference image. This decoded image R may be before filtering or after filtering. The prediction unit 419 supplies the derived predicted image P to the arithmetic unit 415.

[0290] <Encoding control of LFNST identifier> For the image encoding apparatus 300 configured as described above, the present technology described in <1. Signaling pattern of LFNST identifier> can be applied.

[0291] For example, the inverse orthogonal transformation unit 414 performs the inverse process of LFNST as an inverse secondary transformation. At that time, the inverse orthogonal transformation unit 414 acquires the LFNST identifier (lfnst_idx) supplied as secondary transformation control information (secondary transformation identifier) from the decoding unit 412 (arrow 431). Then, the inverse orthogonal transformation unit 414 performs the inverse secondary transformation according to the LFNST identifier.

[0292] That is, the decoding unit 412 supplies the LFNST identifier (lfnst_idx) to the inverse orthogonal transformation unit 414 (arrow 431). The decoding unit 412 has an LFNST identifier decoding unit 201 of the decoding apparatus 200 (FIG. 13). For example, when the LFNST identifier is signaled, the LFNST identifier decoding unit 201 decodes the bit stream to generate the LFNST identifier and supplies it to the inverse orthogonal transformation unit 414. When the LFNST identifier is not signaled, the LFNST identifier decoding unit 201 omits the decoding related to the LFNST identifier, estimates its value as "0", and supplies the LFNST identifier with the value "0" to the inverse orthogonal transformation unit 414.

[0293] At that time, the LFNST identifier decoder 201 determines whether or not an LFNST identifier has been signaled (whether or not to perform decoding related to the LFNST identifier) ​​based on the signaled transformation information Tinfo. For example, the decoder 412 decodes the bitstream to generate the transformation information Tinfo. The LFNST identifier decoder 201 acquires various parameters used for decoding related to the LFNST identifier, which are included in the transformation information Tinfo. For example, the LFNST identifier decoder 201 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionSplitType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are those described in the above-mentioned non-patent literature (eg, non-patent literature 1).

[0294] The decoder 412 generates these parameters by decoding the bitstream. The LFNST identifier decoder 201 then uses these parameters to control the decoding of the LFNST identifier (lfnst_idx), as explained in <1. Signaling pattern of LFNST identifier>.

[0295] For example, the decoding unit 412 controls the decoding related to this LFNST identifier by applying the above-mentioned method 1. That is, if the transform skip flag of at least one component among the valid components in the image decoding is true, the decoding unit 412 omits decoding related to the secondary transform control information, which is control information related to the secondary transform, and estimates that the value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform, and if the transform skip flags of all components are false, decodes the bitstream to generate secondary transform control information.

[0296] For example, the decoder 412 controls the decoding of the LFNST identifier in accordance with the syntax table shown in Fig. 4. Alternatively, the decoder 412 may control the decoding of the LFNST identifier in accordance with the syntax table shown in Fig. 6. By following these syntax tables, the decoder 412 can control the decoding of the LFNST identifier according to the tree type and conversion skip flag, as in the table shown in Fig. 3. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0297] The decoding unit 412 may control the decoding related to this LFNST identifier by applying the above-mentioned method 2. That is, if the transform skip flags of all components among the components valid in the image decoding are true, the decoding unit 412 may omit decoding related to the secondary conversion control information and estimate that the value of the secondary conversion control information is "0." Also, if the transform skip flag of at least one component among the components valid in the image decoding is false, the decoding unit 412 may perform decoding related to the secondary conversion control information.

[0298] In this way, the decoder 412 can control decoding related to the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 8. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0299] Furthermore, the decoding unit 412 may control the decoding related to this LFNST identifier by applying the above-mentioned method 3. That is, if the tree type of the image decoding is dual tree chroma and the transform skip flags of all components among the components valid in the dual tree chroma are true, the decoding unit 412 may skip (omit) the decoding related to the secondary transform control information and estimate that the value of the secondary transform control information is "0." Furthermore, if the transform skip flag of at least one component among the components valid in the image decoding is false, the decoding unit 412 may perform the decoding related to the secondary transform control information.

[0300] In this way, the decoder 412 can control decoding related to the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 9. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0301] Furthermore, the decoding unit 412 may control the decoding related to this LFNST identifier by applying the above-mentioned method 4. That is, if the tree type of the image decoding is dual tree chroma and the transform skip flag of at least one component among the components valid in the dual tree chroma is true, the decoding unit 412 may omit decoding related to the secondary transform control information and estimate that the value of the secondary transform control information is "0." Furthermore, if the transform skip flags of all components are false, the decoding unit 412 may perform decoding related to the secondary transform control information.

[0302] In this way, the decoder 412 can control decoding related to the LFNST identifier according to the tree type and conversion skip flag, as shown in the table in Fig. 10. The control method for each tree type is as described above in <1. Signaling pattern of LFNST identifier>.

[0303] In any of methods 1 to 4, the decoding unit 412 can skip decoding for the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb component and the Cr component of the image are true. Therefore, the image decoding device 400 can suppress a decrease in encoding efficiency.

[0304] <Configuration example> In addition, these processing units (accumulation buffer 411 to prediction unit 419) have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other, and for example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0305] <Flow of image decoding process> Next, an example of the flow of image decoding processing executed by the image decoding device 400 configured as above will be described with reference to the flowchart in FIG.

[0306] When the image decoding process starts, in step S401, the accumulation buffer 411 acquires and holds (accumulates) encoded data (bit stream) supplied from outside the image decoding device 400.

[0307] In step S402, the decoding unit 412 decodes the coded data (bit stream) to obtain the quantized transform coefficient level level. Furthermore, the decoding unit 412 parses (analyzes and obtains) various coding parameters from the coded data (bit stream) through this decoding.

[0308] In step S403, the inverse quantization unit 413 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S402 to obtain a transform coefficient Coeff_IQ.

[0309] In step S404, the inverse orthogonal transform unit 414 performs an inverse orthogonal transform process, which is an inverse process of the orthogonal transform process performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S403, to obtain a prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficient Coeff_IQ (secondary transform coefficient) to generate a primary transform coefficient, and performs an inverse primary transform on the primary transform coefficient to generate a prediction residual D'.

[0310] In step S405, the prediction unit 419 performs a prediction process using a prediction method specified by the encoding side based on the information parsed in step S402, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 418.

[0311] In step S406, the calculation unit 415 adds the prediction residual D' obtained in step S404 and the predicted image P obtained in step S405 to derive a locally decoded image Rlocal.

[0312] In step S407, the in-loop filter unit 416 performs an in-loop filter process on the locally decoded image Rlocal obtained by the process in step S406.

[0313] In step S408, the rearrangement buffer 417 derives the decoded image R using the filtered local decoded image Rlocal obtained by the process of step S407, and rearranges the order of the decoded image R group from the decoding order to the playback order. The decoded image R group rearranged in the playback order is output outside the image decoder 400 as a moving image.

[0314] Also, in step S409, the frame memory 418 stores at least one of the local decoded image Rlocal obtained by the process of step S406 and the filtered local decoded image Rlocal obtained by the process of step S407.

[0315] When the process of step S409 ends, the image decoding process ends.

[0316] <Decoding control regarding the LFNST identifier> The present technology described in <1. Signaling pattern of the LFNST identifier> can be applied to the image decoding process with the above flow. For example, in step S402, the decoding unit 412 (the LFNST identifier decoding unit 201 thereof) decodes the bit stream to generate the LFNST identifier. Then, in step S404, the inverse orthogonal transform unit 414 performs an inverse secondary transform according to the LFNST identifier.

[0317] At that time, the decoding unit 412 controls the decoding regarding the LFNST identifier based on the conversion information Tinfo. That is, as one of the processes in step S402, the decoding unit 412 performs the decoding process shown in the flowchart of FIG. 14.

[0318] For example, in step S201, the decoding unit 412 derives a decoding condition "condition" for the LFNST identifier based on the various parameters included in the conversion information Tinfo described above. In step S202, the decoding unit 412 determines whether the decoding condition "condition" is satisfied. If it is determined that the decoding condition "condition" is true, it means that the LFNST identifier has been signaled, and in step S203, the decoding unit 412 decodes the bitstream and obtains the signaled LFNST identifier (lfnst_idx).

[0319] Furthermore, if it is determined in step S202 that the decoding condition "condition" derived in step S201 is false, the LFNST identifier was not signaled, so the decoding unit 412 skips (omits) decoding of the LFNST identifier (lfnst_idx) in step S204. In other words, if the decoding condition "condition" is not satisfied, the decoding unit 412 skips decoding of the LFNST identifier (lfnst_idx) and estimates that the value of the LFNST identifier is "0" (lfnst_idx = 0).

[0320] In the above decoding process, the decoding unit 412 (the LFNST identifier decoding unit 201) applies, for example, method 1 described above in <1. LFNST identifier signaling pattern>. That is, if the transform skip flag of at least one component among the valid components in image decoding is true, the decoding unit 412 omits decoding of the secondary transform control information, which is control information related to the secondary transform, and if the transform skip flags of all components are false, performs decoding of the secondary transform control information.

[0321] In that case, in step S201, the decoding unit 412 derives the variable NonTsUsed shown in Fig. 5, and uses the derived variable NonTsUsed to derive the coding condition condition shown in Fig. 5. The decoding unit 412 may also derive the variable TsUsed shown in Fig. 7, and uses the derived variable TsUsed to derive the coding condition condition shown in Fig. 7. The decoding unit 412 may also apply any one of methods 2 to 4 to derive the coding condition condition.

[0322] By performing the image decoding process in this manner, the decoding unit 412 can skip decoding related to the LFNST identifier when the tree type is dual tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb component and the Cr component of the image are true, regardless of which method is applied among methods 1 to 4. Therefore, the image decoding device 400 can suppress a decrease in encoding efficiency.

[0323] <Supplementary Information> Note that the image decoding device 400 can apply any inverse coefficient transform as the inverse secondary transform. Therefore, the decoding unit 412 may perform decoding on any secondary transform identifier (any secondary transform control information) without being limited to the example of the LFNST identifier. Also, the above-mentioned methods 1 to 4 can be applied in a suitable combination of some or all of them.

[0324] <6. Notes> <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0325] FIG. 19 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0326] In a computer 800 shown in FIG. 19, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0327] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0328] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0329] In the computer configured as above, the CPU 801 performs the above-mentioned series of processes by, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data and the like necessary for the CPU 801 to execute various processes as appropriate.

[0330] The program executed by the computer can be applied by recording it on a removable medium 821 such as a package medium, for example. In this case, the program can be installed in the storage unit 813 via the input / output interface 810 by mounting the removable medium 821 on the drive 815.

[0331] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0332] In addition, this program can also be installed in advance in the ROM 802 or the storage unit 813.

[0333] <Target of application of this technology> This technology can be applied to any image encoding / decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding / decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. In addition, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0334] This technology can also be applied to a multi-viewpoint image encoding / decoding system that encodes and decodes multi-viewpoint images that include images from multiple views. In that case, this technology can be applied to the encoding / decoding of each view.

[0335] Furthermore, this technology can be applied to a hierarchical image coding (scalable coding) and decoding system that performs coding and decoding of hierarchical images that are layered to have a scalability function for a predetermined parameter. In that case, this technology can be applied to the coding and decoding of each layer.

[0336] Furthermore, in the above, the encoding device 100, the decoding device 200, the image encoding device 300, and the image decoding device 400 have been described as application examples of the present technology, but the present technology can be applied to any configuration.

[0337] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0338] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0339] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0340] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0341] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.

[0342] For example, the present technology can be applied to systems and devices used for providing ornamental content, etc. Furthermore, for example, the present technology can be applied to systems and devices used for traffic, such as monitoring traffic conditions and automatic driving control. Furthermore, for example, the present technology can be applied to systems and devices used for security. Furthermore, for example, the present technology can be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can be applied to systems and devices that monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife, etc. Furthermore, for example, the present technology can be applied to systems and devices used for sports.

[0343] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.

[0344] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.

[0345] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0346] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0347] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0348] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0349] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0350] Also, for example, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel, or individually at a required timing such as when a call is made. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0351] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0352] The present technology can also be configured as follows. (1) Among the effective components in image coding, If the transform skip flag of at least one component is true, omit encoding of secondary transform control information, which is control information regarding the secondary transform; If the transform skip flags of all components are false, encode the secondary transform control information. Encoding section An image processing device comprising: (2) When a tree type of the image coding is a single tree, the coding unit If at least one of a Y component transform skip flag, which is the transform skip flag of a Y component, a Cb component transform skip flag, which is the transform skip flag of a Cb component, and a Cr component transform skip flag, which is the transform skip flag of a Cr component, is true, omit coding of the secondary transform control information; If the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the secondary transform control information is encoded. An image processing device as described in (1). (3) When a tree type of the image coding is a dual tree luma, the coding unit If a Y component transform skip flag is true, omit coding of the secondary transform control information; If the Y component transform skip flag is false, encode the secondary transform control information. An image processing device according to (1) or (2). (4) When a tree type of the image encoding is a dual tree chroma, the encoding unit If at least one of a Cb component transform skip flag, which is the transform skip flag of a Cb component, and a Cr component transform skip flag, which is the transform skip flag of a Cr component, is true, omits encoding of the secondary transform control information; If the Cb component transform skip flag and the Cr component transform skip flag are false, the secondary transform control information is encoded. An image processing device according to any one of (1) to (3). (5) The secondary conversion control information includes a secondary conversion identifier that is an identifier indicating a type of the secondary conversion. An image processing device according to any one of (1) to (4). (6) The secondary transform is a Low Frequency Non-Separable Transform (LFNST). An image processing device according to any one of (1) to (5). (7) The encoding unit, among the effective components in the image encoding, If the conversion skip flags of all components are true, omit encoding the secondary conversion control information; If the transform skip flag of at least one component is false, encode the secondary transform control information. An image processing device according to any one of (1) to (6). (8) The encoding unit is configured such that a tree type of the image encoding is a dual tree chroma, and among the components valid in the dual tree chroma, If the conversion skip flags of all components are true, omit encoding the secondary conversion control information; If the transform skip flag of at least one component is false, encode the secondary transform control information. An image processing device according to any one of (1) to (7). (9) The encoding unit is configured such that a tree type of the image encoding is a dual tree chroma, and among the components valid in the dual tree chroma, If the conversion skip flag of at least one component is true, omitting encoding of the secondary conversion control information; If the transform skip flags of all components are false, encode the secondary transform control information. An image processing device according to any one of (1) to (8). (10) Among the useful components in image coding, If the transform skip flag of at least one component is true, omit encoding of secondary transform control information, which is control information regarding the secondary transform; If the transform skip flags of all components are false, encode the secondary transform control information. Image processing methods.

[0353] (11) Among the effective components in image decoding, If the conversion skip flag of at least one component is true, omit decoding of secondary conversion control information, which is control information related to secondary conversion, and infer that the value of the secondary conversion control information is a predetermined value indicating skipping of the secondary conversion; If the transform skip flags of all components are false, decode the bitstream to generate the secondary transform control information. Decoding section An image processing device comprising: (12) When a tree type of the image decoding is a single tree, the decoding unit If at least one of a Y component transform skip flag, which is the transform skip flag of a Y component, a Cb component transform skip flag, which is the transform skip flag of a Cb component, and a Cr component transform skip flag, which is the transform skip flag of a Cr component, is true, omit decoding related to the secondary transform control information, and estimate that a value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform; If the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information. The image processing device according to (11). (13) When a tree type of the image decoding is a dual tree luma, if a Y component transform skip flag, which is the transform skip flag of a Y component, is true, the decoding unit omits decoding related to the secondary transform control information, and estimates that a value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform; If the Y component transform skip flag is false, the bitstream is decoded to generate the secondary transform control information. The image processing device according to (11) or (12). (14) When a tree type of the image decoding is a dual tree chroma, the decoding unit If at least one of a Cb component transform skip flag, which is the transform skip flag of a Cb component, and a Cr component transform skip flag, which is the transform skip flag of a Cr component, is true, omits decoding of the secondary transform control information, and estimates that a value of the secondary transform control information is a predetermined value indicating skipping of the secondary transform; If the Cb component transform skip flag and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information. An image processing device according to any one of (11) to (13). (15) The secondary conversion control information includes a secondary conversion identifier that is an identifier indicating a type of the secondary conversion. An image processing device according to any one of (11) to (14). (16) The secondary transform is a Low Frequency Non-Separable Transform (LFNST). An image processing device according to any one of (11) to (15). (17) The decoding unit, among the effective components in the image decoding, If the conversion skip flags of all components are true, omit decoding of the secondary conversion control information, and infer that the value of the secondary conversion control information is a predetermined value indicating skipping of the secondary conversion; If the transform skip flag of at least one component is false, decoding the bitstream to generate the secondary transform control information. An image processing device according to any one of (11) to (16). (18) The decoding unit is configured such that a tree type of the image decoding is a dual tree chroma, and among the components valid in the dual tree chroma, If the conversion skip flags of all components are true, omit decoding of the secondary conversion control information, and infer that the value of the secondary conversion control information is a predetermined value indicating skipping of the secondary conversion; If the transform skip flag of at least one component is false, decoding the bitstream to generate the secondary transform control information. An image processing device according to any one of (11) to (17). (19) The decoding unit is configured such that a tree type of the image decoding is a dual tree chroma, and among the components valid in the dual tree chroma, If the conversion skip flag of at least one component is true, omit decoding of the secondary conversion control information, and infer that the value of the secondary conversion control information is a predetermined value indicating skipping of the secondary conversion; If the transform skip flags of all components are false, decode the bitstream to generate the secondary transform control information. An image processing device according to any one of (11) to (18). (20) Among the effective components in image decoding, If the conversion skip flag of at least one component is true, omit decoding of secondary conversion control information, which is control information related to secondary conversion, and infer that the value of the secondary conversion control information is a predetermined value indicating skipping of the secondary conversion; If the transform skip flags of all components are false, decode the bitstream to generate the secondary transform control information. Image processing methods. [Explanation of symbols]

[0354] 100 encoding device, 101 LFNST identifier encoding unit, 200 decoding device, 201 LFNST identifier decoding unit, 300 image encoding device, 301 control unit, 313 orthogonal transform unit, 315 encoding unit, 400 image decoding device, 412 decoding unit, 414 inverse orthogonal transform unit

Claims

1. A derivation unit that derives a variable indicating that a transform skip is not applied to all components by referring to a transform skip flag and a tree type; a coding unit that omits coding of a secondary transformation identifier that is an identifier indicating a type of secondary transformation when the variable derived by the derivation unit indicates false; An image processing device comprising:

2. The encoding unit performs encoding on the secondary translation identifier when the variable derived by the derivation unit indicates true. The image processing device according to claim 1 .

3. The encoding unit performs encoding on the secondary transform identifier when the tree type is a single tree and when a Y component transform skip flag that is the transform skip flag of a Y component, a Cb component transform skip flag that is the transform skip flag of a Cb component, and a Cr component transform skip flag that is the transform skip flag of a Cr component are all false. The image processing device according to claim 2 .

4. The encoding unit performs encoding on the secondary transform identifier when the tree type is a dual tree luma and when a Y component transform skip flag, which is the transform skip flag of a Y component, is false. The image processing device according to claim 2 .

5. The encoding unit performs encoding on the secondary transform identifier when the tree type is dual tree chroma and when a Cb component transform skip flag that is the transform skip flag of a Cb component and a Cr component transform skip flag that is the transform skip flag of a Cr component are false. The image processing device according to claim 2 .

6. The secondary transform is a Low Frequency Non-Separable Transform (LFNST). The image processing device according to claim 1 .

7. deriving a variable indicating that a transform skip is not applied to all components by referring to the transform skip flag and the tree type; omitting coding for a secondary transformation identifier, which is an identifier indicating a type of secondary transformation, when the derived variable indicates false. An image processing method comprising:

8. A derivation unit that derives a variable indicating that the transform skip is not applied to all components by referring to the transform skip flag and the tree type; a decoding unit that omits decoding of a secondary transformation identifier, which is an identifier indicating a type of secondary transformation, when the variable derived by the derivation unit indicates false; An image processing device comprising:

9. The decryption unit further estimates that the secondary transformation identifier indicates skipping of the secondary transformation when the variable derived by the derivation unit indicates false. The image processing device according to claim 8.

10. deriving a variable indicating that a transform skip is not applied to all components by referring to the transform skip flag and the tree type; omitting decoding a secondary transformation identifier, which is an identifier indicating a type of secondary transformation, if the derived variable indicates false. An image processing method comprising:

Images