Image processing apparatus and method

Abstract

The present disclosure relates to an image processing apparatus and an image processing method capable of suppressing a reduction in coding efficiency. In a case where a transform skip flag of at least one of active components in image coding is true, encoding of secondary transform control information as control information related to secondary transform is omitted; and in a case where the transform skip flag of all of the active components in image coding is false, the secondary transform control information is encoded. The present disclosure is applicable to, for example, an image processing apparatus, an image coding apparatus, an image decoding apparatus, a transmission apparatus, a reception apparatus, a transmission / reception apparatus, an information processing apparatus, an imaging apparatus, a reproduction apparatus, an electronic apparatus, an image processing method, an information processing method, or the like.

Description

Technical Field

[0001] This disclosure relates to image processing apparatus and methods, and more specifically to image processing apparatus and methods capable of suppressing reductions in coding efficiency. Background Technology

[0002] Previously, coding methods for deriving prediction residuals from moving images, performing coefficient transformations, quantizing, and encoding have been proposed (e.g., Non-Patent Document 1). In the Universal Video Coding (VVC) described in Non-Patent Document 1, there exists a coding tool that performs a Low-Frequency Inseparable Transform (LFNST) as a secondary transformation of the transform coefficients after the primary transformation and further improves energy concentration. Then, secondary transformation control information, serving as control information regarding the secondary transformation, is communicated via signaling (i.e., the secondary transformation control information is encoded and sent to the decoding side).

[0003] For example, the LFNST identifier (lfnst_idx) indicating the LFNST mode has been signaled as secondary transform control information. This LFNST identifier has also been signaled based on the tree type (treeType) and the transform skip flag (transform_skip_flag). The transform skip flag indicates whether coefficient transforms, including first-order and second-order transforms, are skipped (omitted) in image encoding. Note that this transform skip flag also indicates whether the inverse processing (inverse transform) of coefficient transforms in image decoding is skipped (omitted).

[0004] Reference List

[0005] Non-patent literature

[0006] Non-Patent Document 1: Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, “Versatile Video Coding (Draft 7)”, JVET-P2001-vE, ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 / WG 11, 16th meeting of the Joint Video Experts Group (JVET): Geneva, Switzerland, 19 March 2019 to 11 October 2019. Summary of the Invention

[0007] The problem to be solved by the present invention

[0008] However, in the case of the method described in non-Patent Document 1, even when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true (e.g., 1), the LFNST identifier is notified by signaling, which may lead to redundancy. That is, there is a possibility of reduced coding efficiency.

[0009] This disclosure was made in light of the foregoing, and is intended to enable the suppression of reduced coding efficiency.

[0010] Solution to the problem

[0011] An image processing apparatus according to one aspect of the present technology is an image processing apparatus comprising: an encoding unit configured to: omit encoding of secondary transform control information, which is control information related to a secondary transform, when the transform skip flag of at least one of the valid components in the image encoding is true; and to encode the secondary transform control information when the transform skip flag of all components in the valid components in the image encoding is false.

[0012] An image processing method according to one aspect of the present technology is as follows: when the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of secondary transform control information, which is control information related to the secondary transform, is omitted; and when the transform skip flag of all components in the valid components in the image encoding is false, the secondary transform control information is encoded.

[0013] According to another aspect of the present technology, an image processing apparatus includes a decoding unit configured to: omit decoding related to secondary transform control information when the transform skip flag of at least one component in the valid components of the image decoding is true, and estimate that the value of the secondary transform control information is a predetermined value indicating the skipping of the secondary transform, the secondary transform control information being control information related to the secondary transform; and decode the bitstream to generate secondary transform control information when the transform skip flag of all components in the valid components of the image decoding is false.

[0014] Another aspect of the image processing method according to the present technology is as follows: when the transform skip flag of at least one component in the valid components of the image decoding is true, the decoding associated with secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, the secondary transform control information being control information associated with the secondary transform; and when the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate secondary transform control information.

[0015] In an image processing apparatus and image processing method according to one aspect of the present technology, when the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of secondary transform control information, which is control information related to the secondary transform, is omitted; and when the transform skip flag of all components in the valid components in the image encoding is false, the secondary transform control information is encoded.

[0016] In another aspect of the image processing apparatus and image processing method according to the present technology, when the transform skip flag of at least one component in the valid components of the image decoding is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, the secondary transform control information being control information related to the secondary transform; and when the transform skip flag of all components in the valid components of the image decoding is false, the bit stream is decoded to generate the secondary transform control information. Attached Figure Description

[0017] Figure 1 This is a diagram showing an example of a syntax table for LFNST identifiers.

[0018] Figure 2 This is a diagram showing an example of an LFNST identifier based on tree type and transformation skip flag.

[0019] Figure 3 This is a diagram showing an example of an LFNST identifier based on tree type and transformation skip flag.

[0020] Figure 4 This is a diagram showing an example of a syntax table for LFNST identifiers.

[0021] Figure 5 This is a diagram illustrating an example of an encoding condition.

[0022] Figure 6 This is a diagram showing an example of a syntax table for LFNST identifiers.

[0023] Figure 7 This is a diagram illustrating an example of an encoding condition.

[0024] Figure 8 This is a diagram showing an example of an LFNST identifier based on tree type and transformation skip flag.

[0025] Figure 9 This is a diagram showing an example of an LFNST identifier based on tree type and transformation skip flag.

[0026] Figure 10 This is a diagram showing an example of an LFNST identifier based on tree type and transformation skip flag.

[0027] Figure 11 This is a block diagram illustrating a main configuration example of an image encoding device.

[0028] Figure 12 This is a flowchart illustrating an example of the encoding process.

[0029] Figure 13 This is a block diagram illustrating a main configuration example of the decoding device.

[0030] Figure 14 This is a flowchart used to describe an example of the decoding process.

[0031] Figure 15 This is a block diagram illustrating a main configuration example of an image encoding device.

[0032] Figure 16 This is a flowchart illustrating an example of the image encoding process.

[0033] Figure 17 This is a block diagram illustrating a main configuration example of an image decoding device.

[0034] Figure 18 This is a flowchart illustrating an example of the image decoding process.

[0035] Figure 19 This is a block diagram illustrating a typical configuration example of a computer. Detailed Implementation

[0036] In the following text, patterns for implementing the present disclosure (hereinafter referred to as implementation methods) will be described. Note that the descriptions will be given in the following order.

[0037] 1. Signaling mode of LFNST identifier

[0038] 2. First Embodiment (Encoding Device)

[0039] 3. Second Embodiment (Decoding Device)

[0040] 4. Third Embodiment (Image Encoding Device)

[0041] 5. Fourth Embodiment (Image Decoding Device)

[0042] 6. Supplementary Explanation

[0043] <1. Signaling Mode of LFNST Identifier>

[0044] Supporting documents on technical content and terminology, etc.

[0045] The scope of this technology includes not only the content described in the embodiments, but also the content described in the following non-patent documents known at the time of filing, as well as the content of other documents referenced in the following non-patent documents.

[0046] Non-patent literature 1: (as described above)

[0047] Non-Patent Document 2: Jianle Chen, Yan Ye, Seung Hwan Kim, “Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7)”, 16th Meeting of the Joint Video Experts Group (JVET) of JVET-P2002-v1, ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, Geneva, Switzerland, October 1-11, 2019.

[0048] Non-patent document 3: ITU-T H.264 Recommendation (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017

[0049] Non-patent document 4: ITU-T H.265 Recommendation (02 / 18) "High efficiency video coding", February 2018

[0050] That is, the content described in the aforementioned non-patent literature is also used as the basis for determining the supporting claims. For example, even if the examples do not directly describe the quadtree block structure and the quadtree plus binary tree (QTBT) block structure described in the aforementioned non-patent literature, these contents fall within the scope of the disclosure of this technology and satisfy the supporting claims. Furthermore, for example, even if the examples do not directly describe technical terms such as parsing, syntax, and semantics, these technical terms similarly fall within the scope of the disclosure of this technology and satisfy the supporting claims.

[0051] Furthermore, in this specification, unless otherwise specified, the term "block" (not a block indicating a processing unit) used to describe a partial region of an image (picture) or a processing unit indicates any partial region in the picture, and there is no limitation on the size, shape, characteristics, etc. of the block. For example, "block" includes any partial region (processing unit) such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a smallest coding unit (SCU), a coding unit (CU), a largest coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a sub-block, a macroblock, a tile, or a slice described in the above non-patent literature.

[0052] Furthermore, when specifying the size of such a block, the block size can be specified not only directly but also indirectly. For example, the block size may be specified using identification information for identifying the size. Furthermore, for example, the block size may be specified by a ratio or a difference from the size of a reference block (e.g., LCU, SCU, etc.). For example, when information for specifying a block size is transmitted as a syntax element or the like, the information for indirectly specifying the size as described above may be used as the information. With this configuration, the amount of information can be reduced, and coding efficiency can be improved in some cases. Furthermore, the specification of a block size also includes the specification of a range of block sizes (e.g., the specification of a range of allowable block sizes, etc.).

[0053] Furthermore, in this specification, coding includes not only the entire process of transforming an image into a bitstream, but also a part of the process. For example, coding includes not only processing including prediction processing, orthogonal transform, quantization, and arithmetic coding, but also processing collectively referred to as quantization and arithmetic coding, processing including prediction processing, quantization, and arithmetic coding, and the like. Similarly, decoding includes not only the entire process of transforming a bitstream into an image, but also a part of the process. For example, decoding includes not only processing including inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, and prediction processing, but also processing including inverse arithmetic decoding and inverse quantization, processing including inverse arithmetic decoding, inverse quantization, and prediction processing, and the like.

[0054] <LFNST identifier>

[0055] In the Versatile Video Coding (VVC) described in Non-Patent Document 1, there is a coding tool that performs Low-Frequency Non-Separable Transform (LFNST) as a secondary transform on transform coefficients after a primary transform to further improve energy concentration. Then, secondary transform control information, which is control information about the secondary transform, is signaled (that is, the secondary transform control information is encoded by the encoder and transmitted to a decoding side (decoder)).

[0056] For example, the LFNST identifier (lfnst_idx), which serves as an identifier indicating the mode of LFNST, is used as secondary transformation control information via signaling. A value of "0" for the LFNST identifier (lfnst_idx) indicates that the secondary transformation (LFNST) is skipped (omitted). Furthermore, a value of "1" for the LFNST identifier (lfnst_idx) indicates that the secondary transformation according to the first method is applied. Furthermore, a value of "2" for the LFNST identifier (lfnst_idx) indicates that the secondary transformation according to the second method is applied. Note that, as described below, the signaling for the LFNST identifier (lfnst_idx) can be skipped (omitted). In the case of skipping the signaling for the LFNST identifier (lfnst_idx), the decoding side (decoder) estimates this value.

[0057] The decoder can perform an inverse quadratic transform, i.e., the inverse processing of the quadratic transform, based on the LFNST identifier (or an estimated LFNST identifier) ​​signaled from the encoder side, in a mode corresponding to the quadratic transform performed in the encoder. That is, the quadratic transform control information can also be called control information related to the inverse quadratic transform. Furthermore, the LFNST identifier can also be described as an identifier indicating the mode of the inverse processing of the LFNST performed as an inverse quadratic transform.

[0058] By notifying the LFNST identifier via signaling as described above, the encoder can control the inverse quadratic transform at the decoder. That is, the encoder can instruct the decoder to perform the inverse quadratic transform in a mode corresponding to the quadratic transform during encoding.

[0059] Figure 1 An example of a syntax table for the LFNST identifier in VVC, as described in Non-Patent Document 1, is shown. Figure 1 As shown, if the conditions in rows seven and eight from the top of the syntax table are met, the LFNST identifier (lfnst_idx) shown in row nine from the top is notified by signaling. That is, the LFNST identifier has been notified by signaling based on the tree type (treeType) and the value of the transform skip flag (transform_skip_flag). The transform skip flag is a flag indicating whether coefficient transforms, including first-order transforms, second-order transforms, etc., are skipped (omitted) in image encoding. Note that the transform skip flag also indicates whether the inverse processing (inverse transform) of coefficient transforms is skipped (omitted) in image encoding.

[0060] Figure 2 An example is shown illustrating the correspondence between the combination of tree type and transform skip flag and the signaling of the LFNST identifier. Figure 2In this context, the tree type includes three types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Furthermore, transform_skip_flag[0] indicates the transform skip flag for the luminance (Y) component. transform_skip_flag[1] indicates the transform skip flag for the chrominance (Cb) component. transform_skip_flag[2] indicates the transform skip flag for the chrominance (Cr) component. Figure 2 In the table shown, the "0" and "1" in the columns of each transform skip flag indicate the value of the transform skip flag. In this example, a value of "1" indicates true, and a value of "0" indicates false. Note that there is no signaling for the transform skip flag for invalid components (Y, Cb, or Cr) of each tree type. Figure 2 In this context, the value of the transformation skip flag for such invalid components is indicated by "-".

[0061] In this example, the LFNST identifier can be signaled if the value of transform_skip_flag[0] is "0" or if the tree type is DUAL_TREE_CHROMA. Otherwise, the signaling for the LFNST identifier is skipped. In this case, the value of the LFNST identifier is estimated as "0" in the decoder (inferred as 0).

[0062] However, as Figure 2 As shown in the bottom row of the table, even when the tree type is DUAL_TREE_CHROMA and the transform skip flags for both the Cb and Cr components of the image are true (e.g., 1), the LFNST identifier is notified via signaling. As mentioned above, when the transform skip flag is true, the coefficient transform (inverse coefficient transform) is skipped, and therefore the quadratic transform (inverse quadratic transform) is also skipped. In the case of DUAL_TREE_CHROMA, the effective components are Cb and Cr. Therefore, when the transform skip flags for these components are true, the quadratic transform (inverse quadratic transform) is always skipped. Therefore, there is a possibility that the signaling for the LFNST identifier becomes redundant. That is, there is a possibility that coding efficiency is unnecessarily reduced.

[0063] Method 1

[0064] Therefore, if the transformation skip flag of at least one of the valid components is true, the signaling of the LFNST identifier is skipped (Method 1).

[0065] For example, in an image processing method, if the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of the secondary transform control information, which is control information related to the secondary transform, is omitted; and if the transform skip flag of all the valid components in the image encoding is false, the secondary transform control information is encoded.

[0066] For example, in an image processing apparatus, an encoding unit is provided that, when the transform skip flag of at least one of the valid components in the image encoding is true, omits the encoding of secondary transform control information, which is control information related to the secondary transform, and encodes the secondary transform control information when the transform skip flag of all components in the valid components in the image encoding is false.

[0067] For example, in an image processing method, if the transform skip flag of at least one component in the valid components of the image decoding is true, the decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. The secondary transform control information is control information associated with the secondary transform. If the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0068] For example, the image processing apparatus includes a decoding unit that, when the transform skip flag of at least one component in the valid components of the image decoding is true, omits decoding related to secondary transform control information and estimates that the value of the secondary transform control information is a predetermined value indicating the skipping of the secondary transform, the secondary transform control information being control information related to the secondary transform; and when the transform skip flag of all components in the valid components of the image decoding is false, decodes the bitstream to generate secondary transform control information.

[0069] Note that decoding the secondary transform control information involves decoding the bitstream (also called encoded data) generated by encoding the secondary transform control information to produce the secondary transform control information itself. Furthermore, the predetermined value indicating the skipping of the secondary transform is arbitrary. For example, it can be set to "0".

[0070] Note that the secondary transformation control information can be any information as long as it relates to the control of the secondary transformation, and the secondary transformation control information can include, for example, a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation (and inverse secondary transformation).

[0071] Furthermore, the quadratic transform can be any coefficient transform, and can be, for example, the Low Frequency Inseparable Transform (LFNST). In that case, the inverse quadratic transform can be the inverse process of the LFNST. Moreover, in this case, the quadratic transform identifier is the LFNST identifier (lfnst_idx). Then, the predetermined value indicating the skip of the quadratic transform can be "0" (lfnst_idx == 0).

[0072] Figure 3 An example is shown of the correspondence between the combination of tree type and transformation skip flag and the signaling of LFNST identifier when applying method 1. Figure 3 Is with Figure 2 The table corresponds to the table in the table. The second transformation is LFNST, and the LFNST identifier is notified by signaling as second transformation information. In addition, the predetermined value indicating the skip of the second transformation is assumed to be "0" (lfnst_idx == 0).

[0073] and Figure 2 As in the case of the tree type, there are three types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Furthermore, transform_skip_flag[0] indicates the transform skip flag for the luminance (Y) component. transform_skip_flag[1] indicates the transform skip flag for the chrominance (Cb) component. transform_skip_flag[2] indicates the transform skip flag for the chrominance (Cr) component. The "0" and "1" in the column for each transform skip flag indicate the value of the transform skip flag. In this example, a value of "1" indicates true, and a value of "0" indicates false. Note that there is no signaling for the transform skip flag (indicated by "-") for invalid components (Y, Cb, or Cr) of each tree type.

[0074] In this example, the LFNST identifier can only be signaled if the transform skip flag for all components in the valid components is false (value "0"), regardless of the tree type. If the transform skip flag for at least one component in the valid components is true (value "1"), the signaling for the LFNST identifier is skipped (omitted). In this case, on the decoding side (decoder), the value of the LFNST identifier is estimated to be "0" (i.e., the value indicating a skip of the secondary transform) (inferred as 0).

[0075] Thus, when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the signaling of the LFNST identifier can be skipped. Figure 3(The row at the bottom of the table). Therefore, even if the signaling of the LFNST identifier is skipped in this way, the decoder can still skip the inverse quadratic transform based on the LFNST identifier. That is, the decoder can obtain an LFNST identifier with a value that does not contradict the transform skip flag notified by the signaling, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse quadratic transform in the decoder.

[0076] That is, by applying the above method 1, the redundancy of the signaling of the LFNST identifier can be suppressed, and the reduction in coding efficiency can be suppressed.

[0077] For example, in the case where the tree type for image encoding is single-tree (SINGLE_TREE), Y, Cb, and Cr are valid components. Therefore, in this case, the encoding of the secondary transform control information is skipped when at least one of the Y component transform skip flag, Cb component transform skip flag, and Cr component transform skip flag is true. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). Note that the Y component transform skip flag is the transform skip flag of the Y component (transform_skip_flag[0]). The Cb component transform skip flag is the transform skip flag of the Cb component (transform_skip_flag[1]). The Cr component transform skip flag is the transform skip flag of the Cr component (transform_skip_flag[2]).

[0078] In contrast, when the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag are false, the secondary transform control information can be encoded (yes). That is, the LFNST identifier (lfnst_idx) can be notified by signaling.

[0079] Furthermore, for example, if the image decoding tree type is single-tree and at least one of the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag is true, the decoding of the secondary transform control information is skipped. Then, the value of the secondary transform control information is estimated as a predetermined value indicating the skipping of the secondary transform. That is, in this case, the decoding of the LFNST identifier (lfnst_idx) is skipped. Then, the value of the LFNST identifier is estimated to be "0" (inferred to be 0). In contrast, when the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag are false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0080] Note that in Figure 2In the example case, if the tree type is a single tree and the Y component transform skip flag is false (transform_skip_flag[0] == 0), the LFNST identifier (lfnst_idx) is notified by signaling.

[0081] In contrast, Figure 3 In the example case, if the tree type is a single tree and the Y component transform skip flag is false, and furthermore, if the Cb component transform skip flag and the Cr component transform skip flag are not false (transform_skip_flag[1] == 0 && transform_skip_flag[2] == 0), the signaling of the LFNST identifier (lfnst_idx) is skipped. Therefore, the decoder can suppress the increase in buffer capacity. In addition, the increase in decoding processing latency can be suppressed.

[0082] For example, in the case of image encoding using a dual-tree luminance (DUAL_TREE_LUMA) tree type, luminance Y is the effective component. Therefore, in this case, when the Y component transform skip flag is true, the encoding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). In contrast, when the Y component transform skip flag is false, the secondary transform control information can be encoded (yes). That is, the LFNST identifier (lfnst_idx) is notified by signaling.

[0083] Furthermore, for example, in the case where the tree type for image decoding is dual-tree luminance, and when the Y component transform skip flag is true, decoding of the secondary transform control information is skipped. The value of the secondary transform control information is then estimated to be a predetermined value indicating the skipping of the secondary transform. That is, in this case, decoding of the LFNST identifier (lfnst_idx) is skipped. The value of the LFNST identifier is then estimated to be "0" (inferred to be 0). In contrast, when the Y component transform skip flag is false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0084] For example, in the case where the image encoding tree type is dual-tree chroma (DUAL_TREE_CHROMA), chroma Cb and Cr are valid components. Therefore, in this case, the encoding of secondary transform control information is skipped when at least one of the Cb component transform skip flag or the Cr component transform skip flag is true. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). In contrast, when both the Cb component transform skip flag and the Cr component transform skip flag are false, the secondary transform control information can be encoded (yes). That is, the LFNST identifier (lfnst_idx) is notified by signaling.

[0085] Furthermore, for example, if the image decoding tree type is dual-tree chroma and at least one of the Cb component transform skip flag and the Cr component transform skip flag is true, the decoding of the secondary transform control information is omitted. Then, the value of the secondary transform control information is estimated as a predetermined value indicating the skipping of the secondary transform. That is, in this case, the decoding of the LFNST identifier (lfnst_idx) is skipped. Then, the value of the LFNST identifier is estimated to be "0" (inferred to be 0). In contrast, when the Cb component transform skip flag and the Cr component transform skip flag are false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0086] Figure 4 An example of a syntax table for the LFNST identifier is shown when Method 1 is applied to the VVC described in Non-Patent Document 1. In this case, the encoder (or decoder) sets the variable nonTsUsed for each valid component of the tree type, as follows: Figure 4 As shown at the top. The variable nonTsUsed is an indicator of whether transform skipping is not applied. A true value (e.g., "1") indicates that transform skipping is not applied. A false value (e.g., "0") indicates that transform skipping is applied.

[0087] For example, when the tree type is dual-tree luminance and the Y-component transform skip flag is false (treeType == DUAL_TREE_LUMA && transform_skip_flag[x0][y0][0] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). In contrast, when the Y-component transform skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0088] Furthermore, when the tree type is dual-tree chroma and the Cb component transform skip flag and Cr component transform skip flag are false (treeType == DUAL_TREE_CHROMA && transform_skip_flag[x0][y0][1] == 0 && transform_skip_flag[x0][y0][2] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). In contrast, when either the Cb component transform skip flag or the Cr component transform skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0089] Furthermore, when the tree type is single-tree and the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag are false (treeType == SINGLE_TREE && transform_skip_flag[x0][y0][0] == 0 && transform_skip_flag[x0][y0][1] == 0 && transform_skip_flag[x0][y0][2] == 0), the variable nonTsUsed is set to true (nonTsUsed == 1). In contrast, when at least one of the Y-component transform skip flag, Cb-component transform skip flag, or Cr-component transform skip flag is true, the variable nonTsUsed is set to false (nonTsUsed == 0).

[0090] Then, the variable nonTsUsed is used as one of the conditions to control whether to notify the LFNST identifier (lfnst_idx) via signaling. For example, if the variable nonTsUsed is true (nonTsUsed == 1) and other conditions are met, the encoder encodes the LFNST identifier (lfnst_idx). Furthermore, the decoder also decodes the LFNST identifier (lfnst_idx).

[0091] Furthermore, when the variable nonTsUsed is false (nonTsUsed == 0), the encoder skips the encoding of the LFNST identifier (lfnst_idx). Additionally, the decoder skips the decoding of the LFNST identifier (lfnst_idx) and estimates its value to be "0".

[0092] Figure 5 It is shown that... Figure 4The syntax table is equivalent to a graph of the encoding (decoding) conditions of LFNST identifiers. For example, the encoder sets the variable NonTsUsed for each tree type. Figure 5 (The upper side of the middle). Then, the encoder uses the variable NonTsUsed as one of the conditions to set the encoding condition of the LFNST identifier (condition). Figure 5 (Lower side of the encoder). Then, if the encoding condition is true, the encoder notifies the LFNST identifier (lfnst_idx) with signaling. If the encoding condition is false, the encoder skips the signaling of the LFNST identifier (lfnst_idx).

[0093] For example, the decoder sets the variable NonTsUsed for each tree type. Figure 5 (The upper side of the middle). Then, the decoder uses the variable NonTsUsed as one of the conditions to set the decoding condition of the LFNST identifier (condition). Figure 5 (The lower part of the middle). Then, if the decoding condition is true, the decoder decodes the bitstream to obtain the LFNST identifier (lfnst_idx) notified by signaling. If the decoding condition is false, the decoder skips the decoding of the LFNST identifier (lfnst_idx) and estimates the value of the LFNST identifier (lfnst_idx) to be "0".

[0094] In this way, the reduction in coding efficiency in the VVC described in Non-Patent Document 1 can be suppressed.

[0095] Note that negation expressions can be used in conditional expressions, such as... Figure 6 The syntax table shown. Figure 6 The syntax table shown corresponds to Figure 4 The syntax table is shown, but the variable TsUsed is used instead. Figure 4 The variable `nonTsUsed` is used in the variable TsUsed. `TsUsed` is a variable indicating whether transform skipping was applied; a true value (e.g., "1") indicates that transform skipping was applied, while a false value (e.g., "0") indicates that transform skipping was not applied.

[0096] For example, the variable TsUsed is set as follows: Figure 6As shown at the top of the table. That is, when at least one of the Y-component transform skip flag, Cb-component transform skip flag, or Cr-component transform skip flag is true (transform_skip_flag[x0][y0][0]==1||transform_skip_flag[x0][y0][1]==1||transform_skip_flag[x0][y0][2]==1), the variable TsUsed is set to true (TsUsed==1). Conversely, when the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag are false, the variable TsUsed is set to false (TsUsed==0).

[0097] Then, the variable TsUsed is used as one of the conditions to control whether to notify the LFNST identifier (lfnst_idx) by signaling. For example, if the variable TsUsed is false (TsUsed == 0) and other conditions are met, the encoder encodes the LFNST identifier (lfnst_idx) and the decoder decodes the LFNST identifier (lfnst_idx).

[0098] Furthermore, when the variable TsUsed is true (TsUsed == 1), the encoder skips the encoding of the LFNST identifier (lfnst_idx). Then, the decoder skips the decoding of the LFNST identifier (lfnst_idx) and estimates the value of the LFNST identifier (lfnst_idx) to be "0".

[0099] Figure 7 It is shown that... Figure 6 The syntax table is equivalent to the LFNST identifier's encoding (decoding) condition graph. For example, the encoder sets the variable TsUsed( Figure 7 (The upper side of the middle). Then, the encoder uses the variable TsUsed as one of the conditions to set the encoding condition of the LFNST identifier (condition). Figure 7 (Lower side of the encoder). Then, if the encoding condition is true, the encoder notifies the LFNST identifier (lfnst_idx) with signaling. If the encoding condition is false, the encoder skips the signaling of the LFNST identifier (lfnst_idx).

[0100] For example, the decoder sets the variable TsUsed( Figure 7 (The upper side of the middle). Then, the decoder uses the variable TsUsed as one of the conditions to set the decoding condition of the LFNST identifier (condition). Figure 7(The lower part of the middle). Then, if the decoding condition is true, the decoder decodes the bitstream to obtain the LFNST identifier (lfnst_idx) notified by signaling. If the decoding condition is false, the decoder skips the decoding of the LFNST identifier (lfnst_idx) and estimates the value of the LFNST identifier (lfnst_idx) to be "0".

[0101] In this way, the reduction in coding efficiency in the VVC described in Non-Patent Document 1 can be suppressed.

[0102] Method 2

[0103] For example, if the transform skip flag for all components in the valid components is true, the signaling of the LFNST identifier can be skipped (Method 2).

[0104] For example, if the transform skip flag of all components in the valid components of the image coding is true, the encoding of the secondary transform control information can be omitted, and if the transform skip flag of at least one component in the valid components of the image coding is false, the secondary transform control information can be encoded.

[0105] Furthermore, for example, if the transform skip flag of all components in the valid components of the image decoding is true, the decoding associated with the secondary transform control information can be omitted, and the value of the secondary transform control information can be estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of at least one component in the valid components of the image decoding is false, the bitstream can be decoded to generate the secondary transform control information.

[0106] Even in Method 2, similar to Method 1, the decoding of the secondary transformation control information involves decoding the bitstream generated by encoding the secondary transformation control information to produce the secondary transformation control information itself. Furthermore, the predetermined value indicating the skipping of the secondary transformation is arbitrary.

[0107] Furthermore, the secondary transformation control information can be any information, as long as it pertains to the control of a secondary transformation. For example, the secondary transformation control information may include a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation (and inverse secondary transformation).

[0108] Furthermore, the quadratic transform can be any coefficient transform. For example, the quadratic transform can be LFNST. In that case, the inverse quadratic transform can be the inverse process of LFNST. Moreover, in this case, the quadratic transform identifier is the LFNST identifier (lfnst_idx). Then, the predetermined value indicating the skip of the quadratic transform can be "0" (lfnst_idx == 0).

[0109] Figure 8 An example is shown of the correspondence between the combination of tree type and transformation skip flag and the signaling of LFNST identifier when applying method 2. Figure 8 Is with Figure 2 The table corresponds to the table in the table. The second transformation is LFNST, and the LFNST identifier is notified by signaling as second transformation information. In addition, the predetermined value indicating the skip of the second transformation is assumed to be "0" (lfnst_idx == 0).

[0110] and Figure 2 As with other tree types, there are three types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Additionally, transform_skip_flag[0] indicates the transform skip flag for the Y component. transform_skip_flag[1] indicates the transform skip flag for the Cb component. transform_skip_flag[2] indicates the transform skip flag for the Cr component. The "0" and "1" in the columns of each transform skip flag indicate the value of the transform skip flag. In this example, a value of "1" indicates true, and a value of "0" indicates false. Note that there is no signaling for the transform skip flag (indicated by "-") for invalid components (Y, Cb, or Cr) of each tree type.

[0111] In this example, if the transform skip flag of at least one of the valid components is false (value "0"), the LFNST identifier can be signaled (if possible), regardless of the tree type. The signaling of the LFNST identifier is skipped (omitted) only if the transform skip flag of all components in the valid components is true (value "1"). In this case, on the decoding side (decoder), the value of the LFNST identifier is estimated to be "0" (i.e., the value indicating a skip of the second transform) (inferred as 0).

[0112] Thus, when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the signaling of the LFNST identifier can be skipped. Figure 8 (The row at the bottom of the table). Therefore, even if the signaling for the LFNST identifier is skipped in this way, the decoder can still skip the inverse quadratic transform based on the LFNST identifier. That is, the decoder can obtain an LFNST identifier with a value that does not contradict the transform skip flag notified by the signaling, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse quadratic transform in the decoder.

[0113] That is, by applying the above method 2, the redundancy of the signaling of the LFNST identifier can be suppressed, and the reduction in coding efficiency can be suppressed.

[0114] For example, in the case of a single-tree image encoding, Y, Cb, and Cr are valid components. Therefore, in this case, when the Y component transform skip flag, Cb component transform skip flag, and Cr component transform skip flag are true, the encoding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). In contrast, when at least one of the Y component transform skip flag, Cb component transform skip flag, or Cr component transform skip flag is false, the secondary transform control information can be encoded (yes). That is, the LFNST identifier (lfnst_idx) is notified by signaling.

[0115] Furthermore, for example, if the image decoding tree type is single-tree and the Y-component transform skip flag, Cb-component transform skip flag, and Cr-component transform skip flag are true, the decoding of the secondary transform control information is skipped. Then, the value of the secondary transform control information is estimated as a predetermined value indicating the skipping of the secondary transform. That is, in this case, the decoding of the LFNST identifier (lfnst_idx) is skipped. Then, the value of the LFNST identifier is estimated to be "0" (inferred to be 0). In contrast, when at least one of the Y-component transform skip flag, Cb-component transform skip flag, or Cr-component transform skip flag is false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0116] Note that in Figure 2 In the example case, where the image decoding tree type is a single tree and the Y component transform skip flag is true (transform_skip_flag[0] == 1), the LFNST signaling identifier (lfnst_idx) has been skipped. Therefore, even if the Cb component transform skip flag and the Cr component transform skip flag are false (transform_skip_flag[1] == 0 || transform_skip_flag[2] == 0), the quadratic transform (LFNST) cannot be applied to the Cb and Cr components. Therefore, the coding efficiency of the Cb and Cr components may be reduced.

[0117] In contrast, Figure 8In the case where at least one of the Y component transform skip flag, Cb component transform skip flag, or Cr component transform skip flag is false, the LFNST identifier can be notified by signaling. That is, when the image decoding tree type is single tree and the Y component transform skip flag is true (transform_skip_flag[0] == 1), and when the Cb component transform skip flag and the Cr component transform skip flag are false (transform_skip_flag[1] == 0 ||transform_skip_flag[2] == 0), the quadratic transform (LFNST) can be applied to the Cb and Cr components. Therefore, the reduction in coding efficiency of the Cb and Cr components can be suppressed.

[0118] For example, in the case of image encoding using a dual-tree luminance (DUAL_TREE_LUMA) tree type, luminance Y is the effective component. Therefore, in this case, when the Y component transform skip flag is true, the encoding of the secondary transform control information is skipped. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). In contrast, when the Y component transform skip flag is false, the secondary transform control information can be encoded (yes). That is, the LFNST identifier (lfnst_idx) is notified by signaling.

[0119] Furthermore, for example, in the case where the tree type for image decoding is dual-tree luminance, and when the Y component transform skip flag is true, decoding of the secondary transform control information is skipped. The value of the secondary transform control information is then estimated to be a predetermined value indicating the skipping of the secondary transform. That is, in this case, decoding of the LFNST identifier (lfnst_idx) is skipped. The value of the LFNST identifier is then estimated to be "0" (inferred to be 0). In contrast, when the Y component transform skip flag is false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0120] For example, in the case where the image encoding tree type is dual-tree chroma (DUAL_TREE_CHROMA), chroma Cb and Cr are valid components. Therefore, in this case, the encoding of the secondary transform control information is skipped when both the Cb component transform skip flag and the Cr component transform skip flag are true. That is, in this case, the signaling of the LFNST identifier (lfnst_idx) is skipped. Then, in the decoder, the value of the LFNST identifier is estimated as "0" (inferred as 0). In contrast, the secondary transform control information can be encoded (yes) when at least one of the Cb component transform skip flag or the Cr component transform skip flag is false. That is, the LFNST identifier (lfnst_idx) is notified by signaling.

[0121] Furthermore, for example, if the image decoding tree type is dual-tree chroma and both the Cb component transform skip flag and the Cr component transform skip flag are true, the decoding of the secondary transform control information is omitted. The value of the secondary transform control information is then estimated as a predetermined value indicating the skipping of the secondary transform. That is, in this case, the decoding of the LFNST identifier (lfnst_idx) is skipped. The value of the LFNST identifier is then estimated to be "0" (inferred to be 0). In contrast, when at least one of the Cb component transform skip flag or the Cr component transform skip flag is false, the bitstream is decoded to generate the secondary transform control information. That is, the LFNST identifier (lfnst_idx) notified by signaling is obtained.

[0122] Method 3

[0123] For example, in the case of a single tree and in the case of a dual-tree luminance, the signaling of the LFNST identifier can be controlled in a manner similar to that described in Non-Patent Document 1, and the signaling of the LFNST identifier can only be skipped (method 3) if the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flag of all components in the effective components of dual-tree chroma is true.

[0124] For example, when the tree type for image encoding is dual-tree chroma, the encoding of secondary transform control information can be skipped if the transform skip flag of all valid components in the dual-tree chroma is true, and the secondary transform control information can be encoded if the transform skip flag of at least one valid component in the dual-tree chroma is false.

[0125] Furthermore, for example, when the tree type for image decoding is dual-tree chroma, if the transform skip flag of all components in the effective components of dual-tree chroma is true, the decoding associated with the secondary transform control information can be skipped, and the value of the secondary transform control information can be estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of at least one component in the effective components of dual-tree chroma is false, the bitstream can be decoded to generate the secondary transform control information.

[0126] Even in method 3, similar to method 1, the decoding of the secondary transformation control information involves decoding the bitstream generated by encoding the secondary transformation control information to generate the secondary transformation control information itself. Furthermore, the predetermined value indicating the skipping of the secondary transformation is arbitrary.

[0127] Furthermore, the secondary transformation control information can be any information, as long as it pertains to the control of a secondary transformation. For example, the secondary transformation control information may include a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation (and inverse secondary transformation).

[0128] Furthermore, the quadratic transform can be any coefficient transform. For example, the quadratic transform can be LFNST. In that case, the inverse quadratic transform can be the inverse process of LFNST. Moreover, in this case, the quadratic transform identifier is the LFNST identifier (lfnst_idx). Then, the predetermined value indicating the skip of the quadratic transform can be "0" (lfnst_idx == 0).

[0129] Figure 9 An example is shown of the correspondence between the combination of tree type and transformation skip flag and the signaling of LFNST identifier when applying method 3. Figure 9 Is with Figure 2 The table corresponds to the table in the table. The second transformation is LFNST, and the LFNST identifier is notified by signaling as second transformation information. In addition, the predetermined value indicating the skip of the second transformation is assumed to be "0" (lfnst_idx == 0).

[0130] and Figure 2As with other tree types, there are three types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Additionally, transform_skip_flag[0] indicates the transform skip flag for the Y component. transform_skip_flag[1] indicates the transform skip flag for the Cb component. transform_skip_flag[2] indicates the transform skip flag for the Cr component. The "0" and "1" in the columns of each transform skip flag indicate the value of the transform skip flag. In this example, a value of "1" indicates true, and a value of "0" indicates false. Note that there is no signaling for the transform skip flag (indicated by "-") for invalid components (Y, Cb, or Cr) of each tree type.

[0131] In this example, the tree type is dual-tree chroma, and the LFNST identifier can be signaled if the transform skip flag of at least one of the valid components in the dual-tree chroma is false. In contrast, the encoding of the LFNST identifier is skipped (omitted) when the transform skip flags of all components are true (value "1"). In this case, on the decoding side (decoder), the value of the LFNST identifier is estimated to be "0" (i.e., the value indicating a skipped secondary transform) (inferred as 0).

[0132] Note that in the case of a single-tree tree and in the case of a dual-tree tree, the signaling of the LFNST identifier is controlled using a method similar to that described in Non-Patent Document 1. That is, the signaling control of the LFNST identifier is similar to... Figure 2 The situation in the table.

[0133] Thus, when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the signaling of the LFNST identifier can be skipped. Figure 9 (The row at the bottom of the table). Therefore, even if the signaling for the LFNST identifier is skipped in this way, the decoder can still skip the inverse quadratic transform based on the LFNST identifier. That is, the decoder can obtain an LFNST identifier with a value that does not contradict the transform skip flag notified by the signaling, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse quadratic transform in the decoder.

[0134] That is, by applying method 3 above, the redundancy of signaling for LFNST identifiers can be suppressed, and the reduction in coding efficiency can be suppressed.

[0135] Method 4

[0136] For example, in the case of a single tree and in the case of a dual-tree luminance, the signaling of the LFNST identifier can be controlled in a manner similar to that described in Non-Patent Document 1, and the signaling of the LFNST identifier can only be skipped if the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flag of at least one of the valid components in dual-tree chroma is true (method 4).

[0137] For example, when the tree type for image encoding is dual-tree chroma, the encoding of secondary transform control information can be skipped if the transform skip flag of at least one of the effective components in dual-tree chroma is true, and the encoding of secondary transform control information can be performed if the transform skip flag of all the effective components in dual-tree chroma is false.

[0138] Furthermore, for example, when the tree type for image decoding is dual-tree chroma, if the transform skip flag of at least one of the valid components in dual-tree chroma is true, decoding associated with secondary transform control information can be skipped, and the value of the secondary transform control information can be estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of all components in the valid components in dual-tree chroma is false, the bitstream can be decoded to generate secondary transform control information.

[0139] Even in method 4, similar to method 1, the decoding of the secondary transformation control information involves decoding the bitstream generated by encoding the secondary transformation control information to generate the secondary transformation control information itself. Furthermore, the predetermined value indicating the skipping of the secondary transformation is arbitrary.

[0140] Furthermore, the secondary transformation control information can be any information, as long as it pertains to the control of a secondary transformation. For example, the secondary transformation control information may include a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation (and inverse secondary transformation).

[0141] Furthermore, the quadratic transform can be any coefficient transform. For example, the quadratic transform can be LFNST. In that case, the inverse quadratic transform can be the inverse process of LFNST. Moreover, in this case, the quadratic transform identifier is the LFNST identifier (lfnst_idx). Then, the predetermined value indicating the skip of the quadratic transform can be "0" (lfnst_idx == 0).

[0142] Figure 10 An example is shown of the correspondence between the combination of tree type and transformation skip flag and the signaling of LFNST identifier when applying method 4. Figure 10 Is with Figure 2The table corresponds to the table in the table. The second transformation is LFNST, and the LFNST identifier is notified by signaling as second transformation information. In addition, the predetermined value indicating the skip of the second transformation is assumed to be "0" (lfnst_idx == 0).

[0143] and Figure 2 As with other tree types, there are three types: SINGLE_TREE, DUAL_TREE_LUMA, and DUAL_TREE_CHROMA. Additionally, transform_skip_flag[0] indicates the transform skip flag for the Y component. transform_skip_flag[1] indicates the transform skip flag for the Cb component. transform_skip_flag[2] indicates the transform skip flag for the Cr component. The "0" and "1" in the columns of each transform skip flag indicate the value of the transform skip flag. In this example, a value of "1" indicates true, and a value of "0" indicates false. Note that there is no signaling for the transform skip flag (indicated by "-") for invalid components (Y, Cb, or Cr) of each tree type.

[0144] In this example, the tree type is dual-tree chroma, and the LFNST identifier can be signaled if the transform skip flag for all valid components in the dual-tree chroma is false. In contrast, the encoding of the LFNST identifier is skipped (omitted) when the transform skip flag for at least one component is true (value "1"). In this case, on the decoding side (decoder), the value of the LFNST identifier is estimated to be "0" (i.e., the value indicating a skipped secondary transform) (inferred as 0).

[0145] Note that, similar to Figure 3 In the cases where the tree type is single-tree and in the cases where the tree type is dual-tree brightness, the signaling of the LFNST identifier is controlled by a method similar to that described in Non-Patent Document 1. That is, the signaling control of the LFNST identifier is similar to... Figure 2 The situation in the table.

[0146] Thus, when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the signaling of the LFNST identifier can be skipped. Figure 10 (The row at the bottom of the table). Therefore, even if the signaling for the LFNST identifier is skipped in this way, the decoder can still skip the inverse quadratic transform based on the LFNST identifier. That is, the decoder can obtain an LFNST identifier with a value that does not contradict the transform skip flag notified by the signaling, and can correctly perform image decoding. In other words, the encoder can correctly control the inverse quadratic transform in the decoder.

[0147] That is, by applying method 4 above, the redundancy of signaling for LFNST identifiers can be suppressed, and the reduction in coding efficiency can be suppressed.

[0148] <Combination>

[0149] Some or all of the methods 1 to 4 above can be appropriately combined and applied.

[0150] <2. First Implementation Method>

[0151] <Encoding device>

[0152] The techniques (methods 1 to 4) described in <1. Signaling Modes of LFNST Identifiers> can be applied to any device. Examples of applications of this technique will be described below. Figure 11 This is a block diagram illustrating an example configuration of an encoding device as one mode of an image processing apparatus to which this technology is applied. Figure 11 The encoding device 100 shown is a device for encoding the LFNST identifier (lfnst_idx). For example, the encoding device 100 performs the encoding by applying context-based adaptive binary arithmetic code (CABAC).

[0153] Notice, Figure 11 It shows the main processing unit, data flow, etc., and Figure 11 Those shown are not necessarily all. That is, in the encoding device 100, there may be those not shown. Figure 11 The processing unit of the block in the middle, or there is a block not shown. Figure 11 The processing of arrows or data streams.

[0154] like Figure 11 As shown, the encoding device 100 includes an LFNST identifier encoding unit 101.

[0155] The LFNST identifier encoding unit 101 performs processing related to the encoding of LFNST identifiers. For example, the LFNST identifier encoding unit 101 acquires an LFNST identifier (lfnst_idx) from an external source. The LFNST identifier encoding unit 101 encodes the acquired LFNST identifier to generate a bitstream. The LFNST identifier encoding unit 101 outputs the generated bitstream to the outside of the encoding device 100. This bitstream is transmitted to the decoding side (decoder) via, for example, a communication medium, or stored in a storage medium.

[0156] Furthermore, the LFNST identifier encoding unit 101 acquires various parameters to be used for encoding the LFNST identifier. For example, the LFNST identifier encoding unit 101 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionFlagType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are those described in the aforementioned non-patent literature (e.g., Non-patent Literature 1).

[0157] Then, the LFNST identifier encoding unit 101 uses these parameters to control the encoding (signaling) of the LFNST identifier (lfnst_idx) as described in <1. Signaling mode of LFNST identifier>.

[0158] For example, the LFNST identifier encoding unit 101 controls the encoding of the LFNST identifier by applying the method 1 described above. That is, when the transform skip flag of at least one of the valid components in the image encoding is true, the LFNST identifier encoding unit 101 omits the encoding of the secondary transform control information, which is control information related to the secondary transform, and encodes the secondary transform control information when the transform skip flag of all components in the valid components in the image encoding is false.

[0159] For example, LFNST identifier encoding unit 101 according to Figure 4 The syntax table shown controls the encoding of LFNST identifiers. Note that the LFNST identifier encoding unit 101 can be configured according to... Figure 6 The syntax tables shown control the encoding of LFNST identifiers. Based on these syntax tables, the LFNST identifier encoding unit 101 can control the encoding of LFNST identifiers according to the tree type and transformation skip flags, such as... Figure 3 The table shown illustrates the control methods for each tree type, as described above in <1. Signaling Modes for LFNST Identifiers>.

[0160] Note that the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier by applying the method 2 described above. That is, the LFNST identifier encoding unit 101 can omit the encoding of the secondary transform control information when the skip flag of all components in the valid components of the image encoding is true, and encode the secondary transform control information when the transform skip flag of at least one component in the valid components of the image encoding is false.

[0161] Thus, the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 8 The table shown illustrates the control methods for each tree type, as described in <1. Signaling Modes for LFNST Identifiers> above.

[0162] Furthermore, the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier by applying the method 3 described above. For example, when the tree type of the image encoding is dual-tree chroma, the LFNST identifier encoding unit 101 can omit the encoding of the secondary transform control information when the transform skip flag of all components in the effective components of the dual-tree chroma is true, and encode the secondary transform control information when the transform skip flag of at least one component in the effective components of the dual-tree chroma is false.

[0163] Thus, the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 9 The table shown illustrates the control methods for each tree type, as described in <1. Signaling Modes for LFNST Identifiers> above.

[0164] Furthermore, the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier by applying the method 4 described above. For example, when the tree type of the image encoding is dual-tree chroma, the LFNST identifier encoding unit 101 can omit the encoding of the secondary transform control information when the transform skip flag of at least one of the effective components in the dual-tree chroma is true, and encode the secondary transform control information when the transform skip flag of all the effective components in the dual-tree chroma is false.

[0165] Thus, the LFNST identifier encoding unit 101 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 10 The table shown illustrates the control methods for each tree type, as described in <1. Signaling Modes for LFNST Identifiers> above.

[0166] In any case where any of methods 1 to 4 are applied, and the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the LFNST identifier encoding unit 101 can skip the LFNST identifier signaling. Therefore, the encoding device 100 can suppress the reduction in encoding efficiency.

[0167] <Encoding Process>

[0168] Next, we will refer to Figure 12 The flowchart in the diagram is an example of the process of encoding processing performed in the encoding device 100.

[0169] When the encoding process begins, the LFNST identifier encoding unit 101 of the encoding device 100 derives the encoding condition for the LFNST identifier in step S101. For example, when applying method 1 described in <1. Signaling Mode of LFNST Identifier>, the LFNST identifier encoding unit 101 omits the encoding of secondary transform control information (which is control information related to the secondary transform) when the transform skip flag of at least one component in the valid components of the image encoding is true; and encodes the secondary transform control information when the transform skip flag of all components in the valid components of the image encoding is false. For example, the LFNST identifier encoding unit 101 derives... Figure 5 The variable NonTsUsed is shown, and the exported variable NonTsUsed is used to export... Figure 5 The encoding condition shown.

[0170] Note that the LFNST identifier encoding unit 101 can be derived. Figure 7 The variable TsUsed is shown, and the exported variable TsUsed is used to derive... Figure 7 The encoding condition is shown. Furthermore, the LFNST identifier encoding unit 101 can derive the encoding condition using any of methods 2 to 4.

[0171] In step S102, the LFNST identifier encoding unit 101 determines whether the encoding condition is met. That is, the LFNST identifier encoding unit 101 determines whether the encoding condition derived in step S101 is true. If the encoding condition is determined to be true, the process proceeds to step S103.

[0172] In step S103, the LFNST identifier encoding unit 101 encodes the LFNST identifier (lfnst_idx) to generate a bitstream. That is, when the encoding condition is met, the LFNST identifier encoding unit 101 encodes the LFNST identifier (lfnst_idx) to generate a bitstream. For example, this bitstream is provided to the decoder via a communication medium. Furthermore, this bitstream can be stored in a storage medium. When the processing in step S103 ends, the encoding process ends.

[0173] Furthermore, in step S102, if it is determined that the encoding condition derived in step S101 is false, the process proceeds to step S104.

[0174] In step S104, the LFNST identifier encoding unit 101 skips (omits) the encoding of the LFNST identifier (lfnst_idx). That is, if the encoding condition is not met, the LFNST identifier encoding unit 101 skips the encoding of the LFNST identifier (lfnst_idx). In other words, in this case, no bitstream of the LFNST identifier is generated (i.e., no signaling is used to notify the LFNST identifier). When the processing in step S104 ends, the encoding process ends.

[0175] By performing the encoding process in this manner, in any case where any of methods 1 to 4 are applied, the LFNST identifier encoding unit 101 can skip the LFNST identifier signaling if the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true. Therefore, the encoding device 100 can suppress the reduction in encoding efficiency.

[0176] <Supplement>

[0177] Note that the encoding device 100 (LFNST identifier encoding unit 101) can encode any example of a quadratic transform identifier (arbitrary quadratic transform control information) instead of an LFNST identifier. That is, the quadratic transform is not limited to LFNST and can be any coefficient transform. Furthermore, some or all of the methods 1 to 4 described above can be appropriately combined and applied.

[0178] <3. Second Implementation Method>

[0179] <Decoding device>

[0180] Figure 13 This is a block diagram illustrating an example configuration of a decoding device as one mode of an image processing apparatus to which this technology is applied. Figure 13The decoding apparatus 200 shown is an apparatus for decoding a bitstream (encoded data) of an LFNST identifier (lfnst_idx). The decoding apparatus 200 performs decoding by applying a decoding method (e.g., CABAC) corresponding to the encoding method of the encoding apparatus 100. For example, the decoding apparatus 200 decodes the bitstream generated by the encoding apparatus 100.

[0181] Notice, Figure 13 It shows the main processing unit, data flow, etc., and Figure 13 Those shown are not necessarily all. That is, in the decoding device 200, there may be those not shown. Figure 13 The processing unit of the block in the middle, or there is a block not shown. Figure 13 The processing of arrows or data streams.

[0182] like Figure 13 As shown, the decoding device 200 includes an LFNST identifier decoding unit 201.

[0183] The LFNST identifier decoding unit 201 performs processing related to the decoding of the LFNST identifier. For example, the LFNST identifier decoding unit 201 acquires a bitstream from an external source. The LFNST identifier decoding unit 201 decodes the acquired bitstream to generate an LFNST identifier (lfnst_idx). The LFNST identifier decoding unit 201 outputs the generated LFNST identifier to the outside of the decoding device 200. The LFNST identifier may be transmitted to the decoding side (decoder) via a communication medium, or stored in a storage medium.

[0184] Furthermore, the LFNST identifier decoding unit 201 acquires various parameters to be used for decoding the LFNST identifier. For example, the LFNST identifier decoding unit 201 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionFlagType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are those described in the aforementioned non-patent literature (e.g., Non-patent Literature 1).

[0185] These parameters are encoded in an encoder (e.g., encoding device 100) and provided as a bitstream (i.e., notified by signaling). Decoding device 200 acquires the bitstream and decodes it to generate these parameters. Note that this decoding can be performed by a decoding unit (not shown) outside of LFNST identifier decoding unit 201, or it can be performed by LFNST identifier decoding unit 201 itself.

[0186] Then, the LFNST identifier decoding unit 201 uses these parameters to control the decoding of the LFNST identifier (lfnst_idx) as described in <1. Signaling Mode of LFNST Identifier>.

[0187] For example, the LFNST identifier decoding unit 201 controls the decoding of the LFNST identifier by applying the method 1 described above. That is, when the transform skip flag of at least one component in the valid components of the image decoding is true, the LFNST identifier decoding unit 201 omits the decoding related to the secondary transform control information and estimates that the value of the secondary transform control information is a predetermined value indicating the skipping of the secondary transform. The secondary transform control information is control information related to the secondary transform. And when the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0188] For example, the LFNST identifier decoding unit 201 according to Figure 4 The syntax table shown controls the decoding of LFNST identifiers. Note that the LFNST identifier decoding unit 201 can be configured according to... Figure 6 The syntax tables shown control the decoding of LFNST identifiers. Based on these syntax tables, the LFNST identifier decoding unit 201 can control the decoding of LFNST identifiers according to the tree type and transform skip flags, such as... Figure 3 The syntax table is shown. The control methods for each tree type are as described above in <1. Signaling Modes of LFNST Identifiers>.

[0189] Note that the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier by applying method 2 described above. That is, when the transform skip flag of all components in the valid components of the image decoding is true, the LFNST identifier decoding unit 201 can omit the decoding related to the secondary transform control information and estimate the value of the secondary transform control information to be "0". Furthermore, when the transform skip flag of at least one component in the valid components of the image decoding is false, the LFNST identifier decoding unit 201 can perform decoding of the secondary transform control information.

[0190] Thus, the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 8 The table shown illustrates the control methods for each tree type, as described in section <1. Signaling Modes for LFNST Identifiers> above.

[0191] Furthermore, the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier by applying the method 3 described above. That is, when the tree type of the image decoding is dual-tree chroma, the LFNST identifier decoding unit 201 can skip (omit) the decoding of the secondary transform control information if the transform skip flag of all components in the valid components of the dual-tree chroma is true, and estimate the value of the secondary transform control information to be "0". Furthermore, when the transform skip flag of at least one component in the valid components of the image decoding is false, the LFNST identifier decoding unit 201 can perform the decoding of the secondary transform control information.

[0192] Thus, the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 9 The table shown illustrates the control methods for each tree type, as described in section <1. Signaling Modes for LFNST Identifiers> above.

[0193] Furthermore, the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier by applying the method 4 described above. That is, when the tree type for image decoding is dual-tree chroma, and the transform skip flag of at least one of the valid components in the dual-tree chroma is true, the LFNST identifier decoding unit 201 can omit the decoding of the secondary transform control information and estimate the value of the secondary transform control information to be "0". Conversely, when the transform skip flags of all components are false, decoding of the secondary transform control information can be performed.

[0194] Thus, the LFNST identifier decoding unit 201 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 10 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0195] In any case where any of methods 1 to 4 are applied, and the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the LFNST identifier decoding unit 201 can skip decoding the LFNST identifier. Therefore, the decoding device 200 can suppress the reduction in coding efficiency.

[0196] <Decoding Process>

[0197] Next, we will refer to Figure 14 The flowchart in the diagram illustrates an example of the decoding process performed in the decoding device 200.

[0198] When decoding begins, the LFNST identifier decoding unit 201 of the decoding device 200 derives the decoding condition of the LFNST identifier in step S201. For example, in the case of applying method 1 described in <1. Signaling Mode of LFNST Identifier>, if the transform skip flag of at least one component in the valid components of the image decoding is true, the LFNST identifier decoding unit 201 omits decoding related to the secondary transform control information, which is control information related to the secondary transform, and decodes the secondary transform control information if the transform skip flag of all components in the valid components of the image decoding is false. For example, the LFNST identifier decoding unit 201 derives... Figure 5 The variable NonTsUsed is shown, and the exported variable NonTsUsed is used to export... Figure 5 The decoding condition shown.

[0199] Note that the LFNST identifier decoding unit 201 can be exported. Figure 7 The variable TsUsed is shown, and the exported variable TsUsed is used to derive... Figure 7 The decoding condition is shown. Furthermore, the LFNST identifier decoding unit 201 can derive the decoding condition using any of methods 2 to 4.

[0200] In step S202, the LFNST identifier decoding unit 201 determines whether the decoding condition is met. That is, the LFNST identifier decoding unit 201 determines whether the decoding condition derived in step S201 is true. If the decoding condition is determined to be true, the process proceeds to step S203.

[0201] In step S203, the LFNST identifier decoding unit 201 decodes the bitstream to generate the LFNST identifier (lfnst_idx). That is, when the decoding condition is met, the LFNST identifier decoding unit 201 decodes the bitstream to generate the LFNST identifier (lfnst_idx). When the processing in step S203 is completed, the decoding process ends.

[0202] Furthermore, in step S202, if it is determined that the decoding condition derived in step S201 is false, the process proceeds to step S204.

[0203] In step S204, the LFNST identifier decoding unit 201 skips (omits) the decoding of the LFNST identifier (lfnst_idx). That is, if the decoding condition is not met, the LFNST identifier decoding unit 201 skips the decoding of the LFNST identifier (lfnst_idx) and estimates the value of the LFNST identifier to be "0" (lfnst_idx = 0). When the processing in step S204 ends, the decoding process ends.

[0204] By performing the decoding process in this manner, in any case where any of methods 1 to 4 are applied, when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components are true, the LFNST identifier decoding unit 201 can skip the decoding of the LFNST identifier. Therefore, the decoding device 200 can suppress the reduction in coding efficiency.

[0205] <Supplement>

[0206] Note that the decoding device 200 (LFNST identifier decoding unit 201) can decode any quadratic transform identifier (any quadratic transform control information) instead of an LFNST identifier. That is, the inverse quadratic transform is not limited to the inverse processing of LFNST and can be any inverse coefficient transform. Furthermore, some or all of the methods 1 to 4 described above can be appropriately combined and applied.

[0207] <4. Third Implementation Method>

[0208] Image encoding device

[0209] Figure 15 This is a block diagram illustrating an example configuration of an image encoding apparatus as a mode of image processing apparatus for applying this technology. Figure 15 The image encoding apparatus 300 shown is an apparatus for encoding image data of a moving image. For example, the image encoding apparatus 300 can encode the image data of a moving image using an encoding method described in any non-patent literature.

[0210] Notice, Figure 15 It shows the main processing units (blocks), data streams, etc., and Figure 15 Those shown are not necessarily all. That is, in the image encoding device 300, there may be elements not shown. Figure 15 The processing unit of the block in the middle is not shown as Figure 15The processing of arrows or data streams.

[0211] like Figure 15 As shown, the image encoding device 300 includes a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transformation unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transformation unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0212] <Control Unit>

[0213] The control unit 301 divides the motion picture data held by the rearrangement buffer 311 into blocks (CU, PU, ​​transform block, etc.) in the processing unit based on the block size in an external processing unit or a pre-specified processing unit. Furthermore, the control unit 301 determines the encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be provided to each block based on, for example, rate-distortion optimization (RDO).

[0214] The details of these encoding parameters will be described below. After determining the above encoding parameters, the control unit 301 provides the encoding parameters to each block. Specifically, the encoding parameters are as follows.

[0215] The header information Hinfo is provided to each block.

[0216] Prediction mode information Pinfo is provided to encoding unit 315 and prediction unit 322.

[0217] Transformation information Tinfo is provided to encoding unit 315, quadrature transformation unit 313, quantization unit 314, inverse quantization unit 317 and inverse quadrature transformation unit 318.

[0218] Filter information Finfo is provided to the in-loop filter unit 320.

[0219] <Rearrangement Buffer>

[0220] Each field of the motion picture data (input image) is input to the image encoding device 300 in the reproduction order (display order). The rearrangement buffer 311 acquires and retains (stores) each input image in its reproduction order (display order). Based on the control of the control unit 301, the rearrangement buffer 311 rearranges the input images in the encoding order (decoding order), or divides the input images into blocks in the processing unit. The rearrangement buffer 311 provides the processed input images to the calculation unit 312. Furthermore, the rearrangement buffer 311 also provides the input images (original images) to the prediction unit 322 and the in-loop filter unit 320.

[0221] <Computational Unit>

[0222] The calculation unit 312 receives the image I corresponding to the block in the processing unit and the prediction image P provided by the prediction unit 322 as inputs, and subtracts the prediction image P from the image I to derive the prediction residual D, as shown in the following expression, and provides the prediction residual D to the orthogonal transformation unit 313.

[0223] D = IP

[0224] Orthogonal Transformation Unit

[0225] The orthogonal transformation unit 313 performs processing related to coefficient transformation. For example, the orthogonal transformation unit 313 obtains the prediction residual D provided by the calculation unit 312. In addition, the orthogonal transformation unit 313 obtains transformation information Tinfo provided by the control unit 301.

[0226] Orthogonal transformation unit 313 performs an orthogonal transformation on the prediction residual D based on the transformation information Tinfo to derive the transformation coefficients Coeff. For example, orthogonal transformation unit 313 performs a first transformation on the prediction residual D to generate first transformation coefficients, and performs a second transformation on the first transformation coefficients to generate second transformation coefficients. Orthogonal transformation unit 313 provides the obtained second transformation coefficients as transformation coefficients Coeff to quantization unit 314.

[0227] Note that orthogonal transformation is an example of coefficient transformation, but is not limited to this example. That is, orthogonal transformation unit 313 can perform arbitrary coefficient transformation on the prediction residual D. In addition, orthogonal transformation unit 313 can perform arbitrary coefficient transformation as both a first-order and a second-order transformation.

[0228] <Quantization Unit>

[0229] The quantization unit 314 performs quantization processing. For example, the quantization unit 314 acquires the transform coefficients Coeff provided by the quadrature transform unit 313. Furthermore, the quantization unit 314 acquires transform information Tinfo provided by the control unit 301. Additionally, the quantization unit 314 scales (quantizes) the transform coefficients Coeff based on the transform information Tinfo. Note that this quantization method is arbitrary. Furthermore, the quantization rate is controlled by the rate control unit 323. The quantization unit 314 provides the quantized transform coefficients (i.e., the quantized transform coefficient level) obtained through quantization to the encoding unit 315 and the inverse quantization unit 317.

[0230] <Encoding Unit>

[0231] Encoding unit 315 performs encoding processing. For example, encoding unit 315 acquires the quantization transform coefficient level (level) provided by quantization unit 314. Furthermore, encoding unit 315 acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) provided by control unit 301. Additionally, encoding unit 315 acquires filter information (e.g., filter coefficients) provided by in-loop filter unit 320. Furthermore, encoding unit 315 acquires information about the optimal prediction mode provided by prediction unit 322 as input.

[0232] The encoding unit 315 performs variable-length encoding (e.g., arithmetic encoding) on ​​the quantized transform coefficient level to generate a bit string (encoded data). Furthermore, the encoding unit 315 derives residual information Rinfo from the quantized transform coefficient level and encodes the residual information Rinfo to generate a bit string.

[0233] Encoding unit 315 includes information about the filter provided to filter information Finfo from in-loop filter unit 320. Furthermore, encoding unit 315 includes information about the optimal prediction mode provided to prediction mode information Pinfo from prediction unit 322. Then, encoding unit 315 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to generate a bit string.

[0234] The encoding unit 315 multiplexes the bit strings of various types of information generated as described above to generate encoded data. The encoding unit 315 provides the encoded data to the accumulation buffer 316.

[0235] <Accumulation Buffer>

[0236] The accumulation buffer 316 temporarily stores the encoded data obtained by the encoding unit 315. At a predetermined time, the accumulation buffer 316 outputs the stored encoded data as a bit stream to the outside of the image encoding apparatus 300. For example, the encoded data can be transmitted to the decoding side via any recording medium, any transmission medium, or any information processing device. That is, the accumulation buffer 316 is also a transmission unit for transmitting encoded data (bit stream).

[0237] <Inverse Quantization Unit>

[0238] The inverse quantization unit 317 performs inverse quantization processing. For example, the inverse quantization unit 317 acquires the quantization transform coefficient level (level) provided by the quantization unit 314. Furthermore, the inverse quantization unit 317 acquires transform information (Tinfo) provided by the control unit 301.

[0239] The inverse quantization unit 317 scales (inverse quantization) the value of the quantized transform coefficient level based on the transform information Tinfo. Note that inverse quantization is the inverse process of quantization performed in the quantization unit 314. The inverse quantization unit 317 provides the transform coefficients Coeff_IQ obtained through inverse quantization to the inverse quadrature transform unit 318.

[0240] <Inverse Orthogonal Transformation Unit>

[0241] The inverse quadrature transformation unit 318 performs processing related to the inverse quadrature transformation. For example, the inverse quadrature transformation unit 318 obtains the transformation coefficients Coeff_IQ provided by the inverse quantization unit 317. In addition, the inverse quadrature transformation unit 318 obtains the transformation information Tinfo provided by the control unit 301.

[0242] The inverse orthogonal transformation unit 318 performs an inverse orthogonal transformation on the transformation coefficients Coeff_IQ based on the transformation information Tinfo to derive the prediction residual D'. Note that the inverse orthogonal transformation is the inverse process of the orthogonal transformation performed in the orthogonal transformation unit 313. For example, the inverse orthogonal transformation unit 318 performs an inverse quadratic transformation on the transformation coefficients Coeff_IQ (quadratic transformation coefficients) to generate primary transformation coefficients. Furthermore, the inverse orthogonal transformation unit 318 performs an inverse primary transformation on the primary transformation coefficients to generate the prediction residual D'. Note that the inverse quadratic transformation is the inverse process of the quadratic transformation performed by the orthogonal transformation unit 313. Furthermore, the inverse primary transformation is the inverse process of the primary transformation performed in the orthogonal transformation unit 313.

[0243] The inverse quadrature transformation unit 318 provides the prediction residual D' obtained through the inverse quadrature transformation to the computation unit 319. Note that since the inverse quadrature transformation unit 318 is similar to the inverse quadrature transformation unit on the decoding side (described below), the description given for the decoding side (described below) can be applied to the inverse quadrature transformation unit 318.

[0244] <Computational Unit>

[0245] The calculation unit 319 uses the prediction residual D' provided by the inverse orthogonal transform unit 318 and the prediction image P provided by the prediction unit 322 as inputs. The calculation unit 319 adds the prediction residual D' and the prediction image P corresponding to the prediction residual D' to derive the local decoded image Rlocal. The calculation unit 319 provides the derived local decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

[0246] <In-loop filter unit>

[0247] The in-loop filter unit 320 performs in-loop filtering processing. For example, the in-loop filter unit 320 uses the locally decoded image Rlocal provided by the computing unit 319, the filter information Finfo provided by the control unit 301, and the input image (original image) provided by the rearrangement buffer 311 as input. Note that the information input to the in-loop filter unit 320 is arbitrary, and information other than the aforementioned information can be input. For example, as needed, information such as prediction mode, motion information, code quantity target value, quantization parameter QP, image type, and block (CU, CTU, etc.) can be input to the in-loop filter unit 320.

[0248] The in-loop filter unit 320 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses the input image (original image) and other input information for filtering as needed.

[0249] For example, the in-loop filter unit 320 applies these four in-loop filters in the following order: bilateral filter, deblocking filter (DBF), adaptive offset filter (Sampling Adaptive Offset (SAO)), and adaptive loop filter (ALF). Note that which filter is applied and in what order is arbitrary and can be chosen appropriately.

[0250] Of course, the filtering process performed by the in-loop filter unit 320 is arbitrary and not limited to the examples above. For example, the in-loop filter unit 320 can apply a Wiener filter, etc.

[0251] The in-loop filter unit 320 provides the filtered local decoded image Rlocal to the frame memory 321. Note that when information about the filter, such as filter coefficients, is sent to the decoding side, the in-loop filter unit 320 provides this filter information to the encoding unit 315.

[0252] <Frame Memory>

[0253] Frame memory 321 performs processing related to the storage of data concerning the image. For example, frame memory 321 uses the locally decoded image Rlocal provided from computing unit 319 and the filtered locally decoded image Rlocal provided from in-loop filter unit 320 as inputs, and maintains (stores) these inputs. Furthermore, frame memory 321 reconstructs and maintains the decoded image R for each picture unit using the locally decoded image Rlocal (by storing the decoded image R in a buffer of frame memory 321). Frame memory 321 provides the decoded image R (or a portion thereof) to prediction unit 322 in response to a request from prediction unit 322.

[0254] <Prediction Unit>

[0255] The prediction unit 322 performs processing related to the generation of a predicted image. For example, the prediction unit 322 acquires prediction mode information Pinfo provided from the control unit 301. In addition, the prediction unit 322 acquires an input image (original image) provided from the reorder buffer 311. Furthermore, the prediction unit 322 acquires a decoded image R (or a part thereof) read from the frame memory 321.

[0256] The prediction unit 322 uses the prediction mode information Pinfo and the input image (original image) to perform prediction processing such as inter prediction or intra prediction. That is, the prediction unit 322 performs prediction by using the decoded image R as a reference image, performs motion compensation processing based on the prediction result, and generates a predicted image P.

[0257] The prediction unit 322 provides the generated predicted image P to the calculation units 312 and 319. In addition, the prediction unit 322 provides the prediction mode selected through the above processing (that is, information about the optimal prediction mode) to the encoding unit 315 as required.

[0258] <Rate Control Unit>

[0259] The rate control unit 323 performs processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of encoded data accumulated in the accumulation buffer 316, so that overflow or underflow does not occur.

[0260] <Encoding Control for LFNST Identifier>

[0261] The technique described in <1. Signaling Mode for LFNST Identifier> can be applied to the image encoding apparatus 300 having the above configuration.

[0262] For example, the orthogonal transform unit 313 performs LFNST as a secondary transform, and provides an LFNST identifier (lfnst_idx) as secondary transform control information (secondary transform identifier) to the encoding unit 315 (arrow 331). It should be noted that the orthogonal transform unit 313 also provides the LFNST identifier (lfnst_idx) to the inverse orthogonal transform unit 318 (arrow 332).

[0263] The encoding unit 315 includes the encoding apparatus 100 ( Figure 11The LFNST identifier encoding unit 101 acquires the LFNST identifier provided by the orthogonal transformation unit 313. The LFNST identifier encoding unit 101 encodes the acquired LFNST identifier to generate a bit string. The encoding unit 315 multiplexes the generated bit string with a bit string of other information to generate encoded data. The encoding unit 315 provides the encoded data to the accumulation buffer 316.

[0264] In addition, the LFNST identifier encoding unit 101 acquires information for control purposes such as... Figure 11 Various parameters of the LFNST identifier encoding shown are provided from the control unit 301 as transformation information Tinfo. For example, the LFNST identifier encoding unit 101 acquires parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionSplitType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are those described in the aforementioned non-patent literature (e.g., Non-patent Literature 1).

[0265] Then, the LFNST identifier encoding unit 101 (i.e., encoding unit 315) uses these parameters to control the encoding (signaling) of the LFNST identifier (lfnst_idx), as described in <1. Signaling Mode of LFNST Identifier>.

[0266] For example, the encoding unit 315 controls the encoding of the LFNST identifier by applying the method 1 described above. That is, the encoding unit 315 omits the encoding of the secondary transform control information when the transform skip flag of at least one of the valid components in the image encoding is true. The secondary transform control information is the control information related to the secondary transform. And when the transform skip flag of all the valid components in the image encoding is false, the secondary transform control information is encoded.

[0267] For example, coding unit 315 according to Figure 4 The syntax table shown controls the encoding of the LFNST identifier. Note that encoding unit 315 can be configured according to... Figure 6 The syntax tables shown control the encoding of LFNST identifiers. Based on these syntax tables, encoding unit 315 can control the encoding of LFNST identifiers according to the tree type and transform skip flags, such as... Figure 3 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0268] Note that the encoding unit 315 can control the encoding of the LFNST identifier by applying the method 2 described above. That is, the encoding unit 315 can omit the encoding of the secondary transform control information when the transform skip flag of all components in the valid components of the image encoding is true, and encode the secondary transform control information when the transform skip flag of at least one component in the valid components of the image encoding is false.

[0269] Thus, encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 8 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0270] Furthermore, the encoding unit 315 can control the encoding of the LFNST identifier by applying the method 3 described above. For example, when the tree type of the image encoding is dual-tree chroma, the encoding unit 315 can omit the encoding of the secondary transform control information when the transform skip flag of all components in the effective components of the dual-tree chroma is true, and encode the secondary transform control information when the transform skip flag of at least one component in the effective components of the dual-tree chroma is false.

[0271] Thus, encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 9 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0272] Furthermore, the encoding unit 315 can control the encoding of the LFNST identifier by applying the method 4 described above. For example, when the tree type of the image encoding is dual-tree chroma, the encoding unit 315 can omit the encoding of the secondary transform control information if the transform skip flag of at least one of the effective components in the dual-tree chroma is true, and encode the secondary transform control information if the transform skip flag of all the effective components in the dual-tree chroma is false.

[0273] Thus, encoding unit 315 can control the encoding of the LFNST identifier according to the tree type and the transformation skip flag, such as... Figure 10 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0274] In any case where any of methods 1 to 4 are applied, and the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the coding unit 315 can skip the signaling of the LFNST identifier. Therefore, the image coding apparatus 300 can suppress the reduction in coding efficiency.

[0275] <Configuration Example>

[0276] Note that these processing units (control unit 301, and rearrangement buffers 311 to rate control units 323) have arbitrary configurations. For example, each processing unit can be configured with logic circuitry to implement the above-described processing. Furthermore, each processing unit may include, for example, a CPU, ROM, RAM, etc., and implement the above-described processing by executing a program using the aforementioned resources. Of course, each processing unit can have both configurations, implementing a portion of the above-described processing through logic circuitry and implementing other processing through program execution. The configurations of the processing units can be independent of each other. For example, some processing units can implement a portion of the above-described processing through logic circuitry, some processing units can implement the above-described processing by executing a program, and some processing units can implement the above-described processing through both logic circuitry and program execution.

[0277] <Image Encoding Process>

[0278] Next, we will refer to Figure 16 The flowchart describes an example of a process performed by the image encoding device 300 with the above configuration.

[0279] When the image encoding process begins, in step S301, the rearrangement buffer 311 is controlled by the control unit 301, and the frames of the input motion image data are rearranged from the display order to the encoding order.

[0280] In step S302, the control unit 301 sets up a processing unit (execution block partitioning) for the input image held by the rearrangement buffer 311.

[0281] In step S303, the control unit 301 determines (sets) the encoding parameters of the input image held by the rearrangement buffer 311.

[0282] In step S304, prediction unit 322 performs prediction processing and generates a prediction image with the best prediction mode, etc. For example, in the prediction processing, prediction unit 322 performs intra-frame prediction to generate a prediction image with the best intra-frame prediction mode, performs inter-frame prediction to generate a prediction image with the best inter-frame prediction mode, and selects the best prediction mode from the prediction images based on the cost function value, etc.

[0283] In step S305, the calculation unit 312 calculates the difference between the input image and the predicted image of the optimal mode selected in step S304 through prediction processing. That is, the calculation unit 312 generates a prediction residual D between the input image and the predicted image. The prediction residual D obtained in this way reduces the amount of data compared to the original image data. Therefore, the amount of data can be compressed compared to encoding the image as is.

[0284] In step S306, the orthogonal transformation unit 313 performs orthogonal transformation processing on the prediction residual D generated by the processing in step S305 to derive the transformation coefficients Coeff. For example, the orthogonal transformation unit 313 performs a first transformation on the prediction residual D to generate first transformation coefficients, and performs a second transformation on the first transformation coefficients to generate second transformation coefficients (transformation coefficients Coeff).

[0285] In step S307, the quantization unit 314 quantizes the transformation coefficient Coeff obtained through the processing in step S306 by using the quantization parameters calculated by the control unit 301, etc., to derive the quantization transformation coefficient level.

[0286] In step S308, the inverse quantization unit 317 uses the characteristics corresponding to the quantization characteristics in step S308 to inverse quantize the quantization transformation coefficient level generated by the processing in step S307, so as to derive the transformation coefficient Coeff_IQ.

[0287] In step S309, the inverse orthogonal transformation unit 318 performs an inverse orthogonal transformation on the transformation coefficients Coeff_IQ obtained through the processing in step S308 using a method corresponding to the orthogonal transformation processing in step S306, to derive the prediction residual D'. For example, the inverse orthogonal transformation unit 318 performs an inverse quadratic transformation on the transformation coefficients Coeff_IQ (quadratic transformation coefficients) to generate primary transformation coefficients. Furthermore, the inverse orthogonal transformation unit 318 performs an inverse primary transformation on the primary transformation coefficients to generate the prediction residual D'.

[0288] Note that this inverse orthogonal transform process is similar to the inverse orthogonal transform process performed on the decoding side. Therefore, the description of the decoding side described below can be applied to the inverse orthogonal transform process in step S309.

[0289] In step S310, the calculation unit 319 adds the predicted image obtained by the prediction processing in step S304 to the prediction residual D' derived by the processing in step S309 to generate a local decoded image.

[0290] In step S311, the in-loop filter unit 320 performs in-loop filtering on the locally decoded image derived from the processing in step S310.

[0291] In step S312, the frame memory 321 stores the locally decoded image derived through the processing in step S301 and the filtered locally decoded image in step S311.

[0292] In step S313, the encoding unit 315 encodes the quantized transform coefficient level obtained through the processing in step S307. For example, the encoding unit 315 encodes the quantized transform coefficient level, which is information about the image, by means of arithmetic coding or the like to generate encoded data. In addition, at this time, the encoding unit 315 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo and transformation information Tinfo). Furthermore, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level and encodes the residual information RInfo.

[0293] In step S314, the accumulation buffer 316 accumulates the encoded data thus obtained, and outputs the encoded data to the outside of the image encoding apparatus 300 as a bitstream, for example. For example, the bitstream is transmitted to the decoding side via a transmission path or a recording medium. In addition, the rate control unit 323 performs rate control as required.

[0294] When the processing in step S314 is completed, the image encoding processing ends.

[0295] <Encoding Control for LFNST Identifier>

[0296] The present technology described in <1. Signaling Mode of LFNST Identifier> can be applied to the image encoding processing having the above flow.

[0297] For example, in step S306, the orthogonal transformation unit 313 performs LFNST as a secondary transformation. The orthogonal transformation unit 313 generates an LFNST identifier (lfnst_idx) as secondary transformation control information (secondary transformation identifier) for the secondary transformation (LFNST).

[0298] In step S313, (the LFNST identifier encoding unit 101 of) the encoding unit 315 encodes the LFNST identifier. At this time, the encoding unit 315 controls the encoding of the LFNST identifier based on the transformation information Tinfo. That is, the encoding unit 315 performs Figure 12 the encoding processing shown in the flowchart as one of the processings of step S313.

[0299] For example, in step S101, the encoding unit 315 derives the encoding condition of the LFNST identifier based on the various parameters included in the transformation information Tinfo. In step S102, the encoding unit 315 determines whether the encoding condition is satisfied. If the encoding condition is determined to be true, the encoding unit 315 encodes the LFNST identifier in step S103 to generate a bit string. The encoding unit 315 multiplexes the generated bit string with a bit string of other information to generate encoded data. That is, in this case, the LFNST identifier is notified by signaling.

[0300] Furthermore, in step S102, if the encoding condition derived in step S101 is determined to be false, the encoding unit 315 skips (omits) the encoding of the LFNST identifier in step S104. That is, in this case, the bit string of the LFNST identifier is not generated. In other words, the LFNST identifier is not notified by signaling.

[0301] In the encoding process described above, encoding unit 315 (LFNST identifier encoding unit 101 of encoding unit 315) applies, for example, method 1 described in <1. Signaling mode of LFNST identifier> above. That is, encoding unit 315 omits the encoding of secondary transform control information (which is control information related to the secondary transform) when the transform skip flag of at least one component in the valid components of the image encoding is true, and encodes the secondary transform control information when the transform skip flag of all components in the valid components of the image encoding is false.

[0302] In this case, in step S101, the encoding unit 315 outputs... Figure 5 The variable NonTsUsed is shown, and the exported variable NonTsUsed is used to export... Figure 5 The encoding condition is shown. Note that encoding unit 315 can be derived from... Figure 7 The variable TsUsed is shown, and the exported variable TsUsed is used to derive... Figure 7 The encoding condition is shown. Furthermore, encoding unit 315 can derive the encoding condition by applying any of methods 2 to 4.

[0303] By performing image coding processing in this manner, in any case where any of methods 1 to 4 are applied, coding unit 315 can skip the signaling of the LFNST identifier when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags of both the Cb and Cr components of the image are true. Therefore, image coding apparatus 300 can suppress the reduction in coding efficiency.

[0304] <Supplement>

[0305] Note that the image encoding device 300 can apply arbitrary coefficient transformations as quadratic transformations. Therefore, the encoding unit 315 can encode examples of arbitrary quadratic transformation identifiers (arbitrary quadratic transformation control information) instead of LFNST identifiers. Furthermore, some or all of the methods 1 to 4 described above can be appropriately combined and applied.

[0306] <5. Fourth Implementation Method>

[0307] <Image Decoding Device>

[0308] Figure 17 This is a block diagram illustrating an example configuration of an image decoding device as one mode of an image processing apparatus to which this technology is applied. Figure 17 The image decoding apparatus 400 shown is an apparatus for encoding coded data of a moving image. For example, the image decoding apparatus 400 can decode the coded data using any of the decoding methods described in the aforementioned non-patent documents. For example, the image decoding apparatus 400 decodes the coded data (bitstream) generated by the image encoding apparatus 300 described above.

[0309] Notice, Figure 17 It shows the main processing units (blocks), data streams, etc., and Figure 17 Those shown are not necessarily all. That is, in the image decoding device 400, there may be those not shown. Figure 17 The processing unit of the block in the middle, or there is a block not shown. Figure 17 The processing of arrows or data streams.

[0310] exist Figure 17 In this image decoding apparatus 400, there are accumulation buffers 411, decoding units 412, inverse quantization units 413, inverse quadrature transform units 414, calculation units 415, in-loop filter units 416, rearrangement buffers 417, frame memory 418, and prediction units 419. Note that the prediction unit 419 includes intra-frame prediction units and inter-frame prediction units (not shown). The image decoding apparatus 400 is an apparatus for generating motion picture data by decoding encoded data (bitstream).

[0311] <Accumulation Buffer>

[0312] The accumulation buffer 411 receives the bit stream input to the image decoding device 400 and holds (stores) the bit stream. For example, the accumulation buffer 411 provides the accumulated bit stream to the decoding unit 412 at a predetermined time or when predetermined conditions are met.

[0313] <Decoding Unit>

[0314] Decoding unit 412 performs processing for decoding the image. For example, decoding unit 412 acquires a bitstream provided from accumulation buffer 411. For example, decoding unit 412 performs variable-length decoding on the syntax value of each syntax element from the bit string according to the definition of the syntax table to derive parameters.

[0315] Parameters derived from syntax elements and their grammatical values ​​include, for example, header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 412 parses (analyzes and acquires) such information from the bitstream. This information will be described below.

[0316] <Header Information Hinfo>

[0317] The header information (Hinfo) includes header information such as the Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and Slice Header (SH). The header information (Hinfo) may also include information defining the following: image size (width PicWidth and height PicHeight), bit depth (luminance bitDepth Y and chrominance bitDepth C), chroma array type (CromaArrayType), maximum / minimum CU size (MaxCUSize / Minimum MinCUSize), maximum / minimum quadtree partitioning depth (MaxQTDepth / Minimum MinQTDepth), maximum / minimum binary tree partitioning depth (MaxBTDepth / Minimum MinBTDepth), maximum transform skip block size (MaxTSSize, also known as the maximum transform skip block size), and on / off flags for each encoding tool (also known as enable flags), etc.

[0318] For example, examples of encoding tool on / off flags included in the header information Hinfo include on / off flags related to the transformation and quantization processes described below. Note that encoding tool on / off flags can also be interpreted as flags indicating the presence of the syntax associated with the encoding tool in the encoded data. Furthermore, a value of 1 (true) for an on / off flag indicates that the encoding tool is available. A value of 0 (false) for an on / off flag indicates that the encoding tool is not available. Note that the interpretation of the flag values ​​can be reversed.

[0319] For example, the header information Hinfo may include the inter-component prediction enable flag (ccp_enabled_flag). The inter-component prediction enable flag is a flag indicating whether inter-component prediction (cross-component prediction (CCP)) is available (also known as CC prediction). For example, if the flag is "1" (true), the flag indicates that inter-component prediction is available. If the flag is "0" (false), the flag indicates that inter-component prediction is not available.

[0320] Note that this CCP is also known as inter-component linear prediction (CCLM or CCLMP).

[0321] <Prediction Model Information Pinfo>

[0322] Prediction mode information Pinfo includes information such as the size of the prediction block (PB) to be processed (PBSize), intra-frame prediction mode information (IPinfo), and motion prediction information (MVinfo).

[0323] Intra-prediction mode information (IPinfo) includes, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode from the 7.3.8.5 CodingUnit syntax of JCTVC-W1005, as well as the lumen intra-prediction mode IntraPredModeY derived from this syntax.

[0324] In addition, the intra-frame prediction mode information IPinfo may include, for example, an inter-component prediction flag (ccp_flag(cclmp_flag)). The inter-component prediction flag (ccp_flag(cclmp_flag)) is a flag indicating whether inter-component linear prediction is applied. For example, ccp_flag == 1 indicates that inter-component prediction is applied, and ccp_flag == 0 indicates that inter-component prediction is not applied.

[0325] In addition, the intra-frame prediction mode information (IPinfo) may include a multi-class linear prediction mode flag (mclm_flag). The multi-class linear prediction mode flag (mclm_flag) is information about linear prediction modes (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether multi-class linear prediction modes are set. For example, "0" indicates Class 1 mode (single-class mode) (e.g., CCLMP), and "1" indicates Class 2 mode (multi-class mode) (e.g., MCLMP).

[0326] In addition, the intra-frame prediction mode information (IPinfo) may include a chroma sample location type identifier (chroma_sample_loc_type_idx). The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier used to identify the type (also known as the chroma sample location type) of the pixel location of the chroma component. For example, if the chroma array type (ChromaArrayType) indicates a 420 format as information about the color format, the chroma sample location type identifier is assigned according to the following expression.

[0327] chroma_sample_loc_type_idx==0: Type2

[0328] chroma_sample_loc_type_idx==1:Type3

[0329] chroma_sample_loc_type_idx==2:Type0

[0330] chroma_sample_loc_type_idx==3: Type1.

[0331] Note that the chroma sample location type identifier (chroma_sample_loc_type_idx) is sent as information about the pixel location of the chroma component (chroma_sample_loc_info()) (i.e., stored in the information about the pixel location of the chroma component).

[0332] In addition, the intra-prediction mode information (IPinfo) may include a chroma MPM identifier (chroma_mpm_idx). The chroma MPM identifier (chroma_mpm_idx) is an identifier that indicates which prediction mode candidate in the chroma intra-prediction mode candidate list (intraPredModeCandListC) will be designated as the chroma intra-prediction mode.

[0333] In addition, the intra-prediction mode information IPinfo can include the lumen intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0334] Motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X = {0,1}, mvd, etc. (see, for example, 7.3.8.6 Prediction Unit Syntax of JCTVC-W1005).

[0335] Of course, the information included in the prediction model information Pinfo is arbitrary and can include information other than those mentioned above.

[0336] <Transformation Information Tinfo>

[0337] The transformation information Tinfo can include, for example, the width size TBWSize and height TBHSize of the transformation block to be processed. Note that the base-2 logarithmic value log2TBWSize can be used instead of the width size TBWSize of the transformation block to be processed. Similarly, the base-2 logarithmic value log2TBHSize can be used instead of the height size TBHSize of the transformation block to be processed.

[0338] In addition, the transformation information Tinfo may include a transformation skip flag (transform_skip_flag (or ts_flag)). The transformation skip flag is a flag that indicates whether to skip coefficient transformations (or inverse coefficient transformations). Note that the transformation skip flag can be signaled for each component of Y, Cb, and Cr (transform_skip_flag[0], transform_skip_flag[1], and transform_skip_flag[2]).

[0339] In addition, the transformation information Tinfo can include parameters such as scan identifier (scanIdx), quantization parameter (qp), and quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 scaling list data syntax)).

[0340] Of course, the information included in the residual information Rinfo is arbitrary and can include information other than the information mentioned above.

[0341] <Residual Information Rinfo>

[0342] The residual information Rinfo (see, for example, section 7.3.8.11 of JCTVC-W1005, Residual Coding Syntax) may include, for example, a residual data presence / absence flag (cbf(coded_block_flag)). Furthermore, the residual information Rinfo may include the last non-zero coefficient X-coordinate (last_sig_coeff_x_pos) and the last non-zero coefficient Y-coordinate (last_sig_coeff_y_pos). Additionally, the residual information Rinfo may include a sub-block non-zero coefficient presence / absence flag (coded_sub_block_flag) and a non-zero coefficient presence / absence flag (sig_coeff_flag).

[0343] Furthermore, the residual information Rinfo may include a GR1 flag (gr1_flag) indicating whether the level of the non-zero coefficient is greater than 1, and a GR2 flag (gr2_flag) indicating whether the level of the non-zero coefficient is greater than 2. Additionally, the residual information Rinfo may include a sign code (sign_flag) indicating the sign of the non-zero coefficient. Furthermore, the residual information Rinfo may include the non-zero coefficient residual level (coeff_abs_level_remaining) representing the level of the non-zero coefficient.

[0344] Of course, the information included in the residual information Rinfo is arbitrary and can include information other than the information mentioned above.

[0345] <Filter Information Finfo>

[0346] The filter information (Finfo) includes control information about the filtering process. For example, Finfo may include control information about the deblocking filter (DBF). Furthermore, Finfo may include control information about the pixel adaptive offset (SAO). Additionally, Finfo may include control information about the adaptive loop filter (ALF). Furthermore, Finfo may include control information about other linear and nonlinear filters.

[0347] For example, filter information (Finfo) can include information about the image to which each filter was applied and the region within the specified image. Furthermore, filter information (Finfo) can include filter on / off control information in units of CUs (cubic units). Additionally, filter information (Finfo) can include filter on / off control information regarding the boundaries of a slice or image.

[0348] Of course, the information included in the filter information Finfo is arbitrary and can include information other than the information mentioned above.

[0349] The description of decoding unit 412 is returned. Decoding unit 412 references the residual information Rinfo and derives the quantized transform coefficient level at each coefficient position in each transform block. Decoding unit 412 provides the quantized transform coefficient level to inverse quantization unit 413.

[0350] In addition, the decoding unit 412 provides each block with the parsed header information Hinfo, prediction mode information Pinfo, quantization transform coefficient level, transform information Tinfo, and filter information Finfo. A detailed description follows.

[0351] The header information Hinfo is provided to the inverse quantization unit 413, the inverse quadrature transform unit 414, the prediction unit 419, and the in-loop filter unit 416. The prediction mode information Pinfo is provided to the inverse quantization unit 413 and the prediction unit 419. The transform information Tinfo is provided to the inverse quantization unit 413 and the inverse quadrature transform unit 414. The filter information Finfo is provided to the in-loop filter unit 416.

[0352] Of course, the above example is merely an example, and this implementation is not limited to this example. For example, each encoding parameter can be provided to any processing unit. Furthermore, other information can be provided to any processing unit.

[0353] <Inverse Quantization Unit>

[0354] The inverse quantization unit 413 performs inverse quantization processing. For example, the inverse quantization unit 413 acquires the transform information Tinfo and the quantization transform coefficient level level provided by the decoding unit 412. Furthermore, the inverse quantization unit 413 scales (inverse quantizes) the value of the quantization transform coefficient level level based on the transform information Tinfo to derive the inverse quantized transform coefficient Coeff_IQ.

[0355] Note that this inverse quantization is performed as the inverse of the quantization performed by the quantization unit 314 of the image encoding device 300. Furthermore, the inverse quantization is a process similar to the inverse quantization performed by the inverse quantization unit 317 of the image encoding device 300. In other words, the inverse quantization unit 317 performs a process similar to that of the inverse quantization unit 413 (inverse quantization).

[0356] The inverse quantization unit 413 provides the derived transform coefficients Coeff_IQ to the inverse orthogonal transform unit 414.

[0357] <Inverse Orthogonal Transformation Unit>

[0358] The inverse quadrature transform unit 414 performs processing related to the inverse quadrature transform. For example, the inverse quadrature transform unit 414 obtains the transform coefficients Coeff_IQ provided by the inverse quantization unit 413. In addition, the inverse quadrature transform unit 414 obtains the transform information Tinfo provided by the decoding unit 412.

[0359] The inverse orthogonal transformation unit 414 performs an inverse orthogonal transformation on the transformation coefficients Coeff_IQ based on the transformation information Tinfo to derive the prediction residual D'. For example, the inverse orthogonal transformation unit 414 performs an inverse quadratic transformation on the transformation coefficients Coeff_IQ to generate first-order transformation coefficients, and performs a first-order transformation on the first-order transformation coefficients to generate the prediction residual D'.

[0360] Note that the inverse orthogonal transformation is performed as the inverse of the orthogonal transformation performed by the orthogonal transformation unit 313 of the image encoding device 300. Furthermore, the inverse orthogonal transformation is a process similar to the inverse orthogonal transformation performed by the inverse orthogonal transformation unit 318 of the image encoding device 300. In other words, the inverse orthogonal transformation unit 318 performs a process similar to that of the inverse orthogonal transformation unit 414 (inverse orthogonal transformation).

[0361] The inverse orthogonal transformation unit 414 provides the derived prediction residual D' to the calculation unit 415.

[0362] <Computational Unit>

[0363] The calculation unit 415 performs processing related to the addition of information about the image. For example, the calculation unit 415 obtains the prediction residual D' provided by the inverse orthogonal transform unit 414. Furthermore, the calculation unit 415 obtains the prediction image P provided by the prediction unit 419. The calculation unit 415 adds the prediction residual D' and the prediction image P (prediction signal) corresponding to the prediction residual D' to derive the local decoded image Rlocal, as shown in the following expression.

[0364] Rlocal=D'+P

[0365] The computation unit 415 provides the exported local decoded image Rlocal to the in-loop filter unit 416 and the frame memory 418.

[0366] <In-loop filter unit>

[0367] The in-loop filter unit 416 performs processing related to in-loop filtering. For example, the in-loop filter unit 416 acquires the local decoded image Rlocal provided by the calculation unit 415. Furthermore, the in-loop filter unit 416 acquires filter information Finfo provided by the decoding unit 412. Note that the information input to the in-loop filter unit 416 is arbitrary, and information other than the aforementioned information can be input.

[0368] The in-loop filter unit 416 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo. For example, the in-loop filter unit 416 applies four in-loop filters in the following order: bilateral filter, deblocking filter (DBF), adaptive offset filter (Sampling Adaptive Offset (SAO)), and adaptive loop filter (ALF). Note that which filter is applied and in what order is applied is arbitrary and can be appropriately chosen.

[0369] The in-loop filter unit 416 performs filtering processing corresponding to that performed on the encoding side (e.g., by the in-loop filter unit 320 of the image encoding device 300). Of course, the filtering processing performed by the in-loop filter unit 416 is arbitrary and not limited to the examples above. For example, the in-loop filter unit 416 may apply a Wiener filter, etc.

[0370] The in-loop filter unit 416 provides the filtered local decoded image Rlocal to the rearrangement buffer 417 and the frame memory 418.

[0371] <Rearrangement Buffer>

[0372] The rearrangement buffer 417 receives the local decoded image Rlocal provided by the in-loop filter unit 416 as input and retains (stores) the local decoded image Rlocal. The rearrangement buffer 417 uses the local decoded image Rlocal to reconstruct the decoded image R for each picture unit and retains (stores) the decoded image R (in the buffer). The rearrangement buffer 417 rearranges the obtained decoded images R from the decoding order to the playback order. The rearrangement buffer 417 outputs the rearranged group of decoded images R to the outside of the image decoding device 400 as moving image data.

[0373] <Frame Memory>

[0374] Frame memory 418 performs processing related to the storage of data about the image. For example, frame memory 418 acquires a local decoded image Rlocal provided by computing unit 415. Then, frame memory 418 uses the local decoded image Rlocal to reconstruct the reconstructed decoded image R for each picture unit. Frame memory 418 stores the reconstructed decoded image R in a buffer within frame memory 418.

[0375] Furthermore, frame memory 418 acquires the in-loop filtered local decoded image Rlocal from in-loop filter unit 416. Then, frame memory 418 uses the in-loop filtered local decoded image to reconstruct the decoded image R for each picture unit. Frame memory 418 stores the reconstructed decoded image R in a buffer within frame memory 418.

[0376] In addition, the frame memory 418 appropriately provides the stored decoded image R (or a portion thereof) as a reference image to the prediction unit 419.

[0377] Note that the frame memory 418 can store header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc., related to the generation of the decoded image.

[0378] <Prediction Unit>

[0379] The prediction unit 419 performs processing related to generation of a predicted image. For example, the prediction unit 419 acquires prediction mode information Pinfo provided from the decoding unit 412. Furthermore, the prediction unit 419 performs prediction through a prediction method specified by the prediction mode information Pinfo to derive a predicted image P. When deriving the predicted image P, the prediction unit 419 uses, as a reference image, a decoded image R (or a portion thereof) stored in the frame memory 418, the decoded image R being specified by the prediction mode information Pinfo. The decoded image R may be an image before filtering or after filtering. The prediction unit 419 provides the derived predicted image P to the arithmetic unit 415.

[0380] <Encoding Control of LFNST Identifier>

[0381] <The present technology described in 1. Signaling Mode of LFNST Identifier> can be applied to the image encoding apparatus 300 having the above configuration.

[0382] For example, the inverse orthogonal transform unit 414 performs inverse processing of LFNST as inverse secondary transform. At this time, the inverse orthogonal transform unit 414 acquires the provided LFNST identifier (lfnst_idx) from the decoding unit 412 as secondary transform control information (secondary transform identifier) (arrow 431). Then, the inverse orthogonal transform unit 414 performs inverse secondary transform in accordance with the LFNST identifier.

[0383] That is, the decoding unit 412 provides the LFNST identifier (lfnst_idx) to the inverse orthogonal transform unit 414 (arrow 431). The decoding unit 412 includes the decoding apparatus 200 ( Figure 13 ) LFNST identifier decoding unit 201. For example, when the LFNST identifier is signaled, the LFNST identifier decoding unit 201 decodes a bitstream to generate the LFNST identifier, and provides the LFNST identifier to the inverse orthogonal transform unit 414. When the LFNST identifier is not signaled, the LFNST identifier decoding unit 201 omits decoding related to the LFNST identifier, estimates the value thereof as "0", and provides the LFNST identifier with the value "0" to the inverse orthogonal transform unit 414.

[0384] At this time, the LFNST identifier decoding unit 201 determines whether the LFNST identifier has been notified by signaling (whether decoding related to the LFNST identifier has been performed) based on the transformation information Tinfo notified by signaling. For example, the decoding unit 412 decodes the bitstream to generate the transformation information Tinfo. The LFNST identifier decoding unit 201 obtains various parameters for decoding related to the LFNST identifier included in the transformation information Tinfo. For example, the LFNST identifier decoding unit 201 obtains parameters such as treeType, transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2], lfnstWidth, lfnstHeight, sps_lfnst_enabled_flag, intra_mip_flag, cbWidth, cbHeight, IntraSubPartitionFlagType, LfnstDcOnly, and LfnstZeroOutSigCoefFlag. These parameters are those described in the aforementioned non-patent literature (e.g., Non-patent Literature 1).

[0385] Decoding unit 412 generates these parameters by decoding the bitstream. Then, LFNST identifier decoding unit 201 uses these parameters to control the decoding of LFNST identifiers (lfnst_idx), as described in <1. Signaling Mode of LFNST Identifier> above.

[0386] For example, the decoding unit 412 controls the decoding of the LFNST identifier by applying the method 1 described above. That is, when the transform skip flag of at least one component in the valid components of the image decoding is true, the decoding unit 412 omits the decoding related to the secondary transform control information, and estimates that the value of the secondary transform control information is a predetermined value indicating the skipping of the secondary transform. The secondary transform control information is control information related to the secondary transform. When the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0387] For example, decoding unit 412 according to Figure 4 The syntax table shown controls the decoding of the LFNST identifier. Note that the decoding unit 412 can be configured according to... Figure 6 The syntax tables shown control the decoding of LFNST identifiers. Based on these syntax tables, decoding unit 412 can control the decoding of LFNST identifiers according to the tree type and transform skip flags, such as... Figure 3 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0388] Note that the decoding unit 412 can control the decoding of the LFNST identifier by applying method 2 described above. That is, when the transform skip flag of all components in the valid components of the image decoding is true, the decoding unit 412 can omit the decoding related to the secondary transform control information and estimate the value of the secondary transform control information to be "0". Furthermore, when the transform skip flag of at least one component in the valid components of the image decoding is false, the decoding unit 412 can perform decoding of the secondary transform control information.

[0389] Thus, the decoding unit 412 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 8 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0390] Furthermore, the decoding unit 412 can control the decoding of the LFNST identifier by applying the method 3 described above. That is, when the tree type for image decoding is dual-tree chroma, and the transform skip flag of all components in the valid components of dual-tree chroma is true, the decoding unit 412 can skip (omit) the decoding of the secondary transform control information and estimate the value of the secondary transform control information to be "0". Furthermore, when the transform skip flag of at least one component in the valid components of the image decoding is false, the decoding unit 412 can perform decoding of the secondary transform control information.

[0391] Thus, the decoding unit 412 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 9 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0392] Furthermore, the decoding unit 412 can control the decoding of the LFNST identifier by applying the method 4 described above. That is, when the tree type for image decoding is dual-tree chroma, if the transform skip flag of at least one of the valid components in the dual-tree chroma is true, the decoding unit 412 can omit decoding the secondary transform control information and estimate the value of the secondary transform control information to be "0". Furthermore, when the transform skip flags of all components are false, the decoding unit 412 can perform decoding of the secondary transform control information.

[0393] Thus, the decoding unit 412 can control the decoding of the LFNST identifier according to the tree type and the transform skip flag, such as... Figure 10 The table shown illustrates the control methods for each tree type, as described in section 1. Signaling Modes for LFNST Identifiers above.

[0394] In any case where any of methods 1 to 4 are applied, and the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true, the decoding unit 412 can skip decoding the LFNST identifier. Therefore, the image decoding apparatus 400 can suppress the reduction in coding efficiency.

[0395] <Configuration Example>

[0396] Note that these processing units (accumulation buffers 411 to prediction units 419) can be configured arbitrarily. For example, each processing unit can be configured with logic circuitry to implement the above-described processing. Furthermore, each processing unit can include, for example, a CPU, ROM, RAM, etc., and implement the above-described processing by executing a program using the aforementioned resources. Of course, each processing unit can have both configurations, implementing a portion of the above-described processing through logic circuitry and implementing other processing through program execution. The configuration of the processing units can be independent of each other. For example, some processing units can implement a portion of the above-described processing through logic circuitry, some processing units can implement the above-described processing by executing a program, and some processing units can implement the above-described processing through both logic circuitry and program execution.

[0397] Image decoding process >

[0398] Next, we will refer to Figure 18 The flowchart describes an example of the image decoding process performed by the image decoding device 400 with the above configuration.

[0399] When the image decoding process begins, in step S401, the accumulation buffer 411 acquires and holds (accumulates) the encoded data (bit stream) provided from outside the image decoding device 400.

[0400] In step S402, the decoding unit 412 decodes the encoded data (bitstream) to obtain the quantization transform coefficient level. Furthermore, the decoding unit 412 parses (analyzes and obtains) various encoding parameters from the encoded data (bitstream) through this decoding.

[0401] In step S403, the inverse quantization unit 413 performs inverse quantization on the quantization transform coefficient level obtained through the processing in step S402 to obtain the transform coefficient Coeff_IQ. This inverse quantization is the inverse processing of quantization performed on the encoding side.

[0402] In step S404, the inverse orthogonal transform unit 414 performs an inverse orthogonal transform on the transform coefficients Coeff_IQ obtained in S403 to obtain the prediction residual D'. This inverse orthogonal transform is the inverse of the orthogonal transform performed on the encoding side. For example, the inverse orthogonal transform unit 414 performs an inverse quadratic transform on the transform coefficients Coeff_IQ (quadratic transform coefficients) to generate first-order transform coefficients, and performs an inverse first-order transform on the first-order transform coefficients to generate the prediction residual D'.

[0403] In step S405, prediction unit 419 performs prediction processing based on the information parsed in step S402 by using the prediction method specified on the encoding side, and generates a prediction image P, for example, by referring to a reference image stored in frame memory 418.

[0404] In step S406, the calculation unit 415 adds the prediction residual D' obtained in step S404 to the prediction image P obtained in step S405 to derive the local decoded image Rlocal.

[0405] In step S407, the inner-loop filter unit 416 performs inner-loop filtering on the local decoded image Rlocal obtained through the processing in step S406.

[0406] In step S408, the rearrangement buffer 417 uses the filtered local decoded image Rlocal obtained through the processing in step S407 to derive the decoded image R, and rearranges the group of decoded images R from the decoding order to the playback order. The group of decoded images R rearranged in playback order is output as a moving image to the outside of the image decoding device 400.

[0407] Furthermore, in step S409, the frame memory 418 stores at least one of the local decoded image Rlocal obtained through the processing in step S406 or the filtered local decoded image Rlocal obtained through the processing in step S407.

[0408] When the processing in step S409 is completed, the image decoding process ends.

[0409] <Control over Decoding the LFNST Identifier>

[0410] The technique described in <1. Signaling Mode of LFNST Identifier> can be applied to image decoding processing with the above-described flow. For example, in step S402, the decoding unit 412 (LFNST identifier decoding unit 201) decodes the bitstream to generate an LFNST identifier. Then, in step S404, the inverse quadrature transform unit 414 performs an inverse quadratic transform based on the LFNST identifier.

[0411] At this time, the decoding unit 412 controls the decoding of the LFNST identifier based on the transformation information Tinfo. That is, the decoding unit 412 executes... Figure 14 The decoding process shown in the flowchart is one of the processes in step S402.

[0412] For example, in step S201, the decoding unit 412 derives the decoding condition of the LFNST identifier based on the various parameters included in the transformation information Tinfo. In step S202, the decoding unit 412 determines whether the decoding condition is met. If the decoding condition is determined to be true, the LFNST identifier is notified by signaling. Therefore, in step S203, the decoding unit 412 decodes the bitstream to obtain the LFNST identifier (lfnst_idx) notified by signaling.

[0413] Furthermore, in step S202, if the decoding condition derived in step S201 is determined to be false, no signaling is used to notify the LFNST identifier. Therefore, in step S204, the decoding unit 412 skips (omits) the decoding of the LFNST identifier (lfnst_idx). That is, if the decoding condition is not met, the decoding unit 412 skips the decoding of the LFNST identifier (lfnst_idx) and estimates the value of the LFNST identifier to be "0" (lfnst_idx = 0).

[0414] In the decoding process described above, the decoding unit 412 (LFNST identifier decoding unit 201 of the decoding unit 412) applies, for example, method 1 described in <1. Signaling mode of LFNST identifier> above. That is, when the transform skip flag of at least one component in the valid components of the image decoding is true, the decoding unit 412 omits the decoding of the secondary transform control information, which is control information related to the secondary transform, and when the transform skip flag of all components in the valid components of the image decoding is false, the decoding unit 412 decodes the secondary transform control information.

[0415] In this case, in step S201, the decoding unit 412 outputs... Figure 5 The variable NonTsUsed is shown, and the exported variable NonTsUsed is used to export... Figure 5 The encoding condition is shown. Note that decoding unit 412 can derive... Figure 7 The variable TsUsed is shown, and the exported variable TsUsed is used to derive... Figure 7The encoding condition is shown. Furthermore, the decoding unit 412 can apply any of methods 2 to 4 to derive the encoding condition.

[0416] By performing image decoding in this manner, in any case where any of methods 1 to 4 are applied, decoding unit 412 can skip decoding of the LFNST identifier when the tree type is dual-tree chroma (DUAL_TREE_CHROMA) and the transform skip flags for both the Cb and Cr components of the image are true. Therefore, image decoding apparatus 400 can suppress the reduction in coding efficiency.

[0417] <Supplement>

[0418] Note that the image decoding device 400 can apply any inverse coefficient transform as an inverse quadratic transform. Therefore, the decoding unit 412 can perform encoding of any quadratic transform identifier (arbitrary quadratic transform control information) instead of encoding examples of the LFNST identifier. Furthermore, some or all of the methods 1 to 4 described above can be appropriately combined and applied.

[0419] <6. Supplementary Explanation>

[0420] Computer

[0421] The above series of processes can be performed by hardware or software. If the processes are performed by software, a program for configuring that software is installed in the computer. Here, "computer" includes computers integrated into dedicated hardware, and computers capable of performing various functions by installing various programs, such as general-purpose personal computers.

[0422] Figure 19 This is a block diagram illustrating an example configuration of the hardware of a computer that performs the above series of processes through a program.

[0423] exist Figure 19 In the computer 800 shown, the central processing unit (CPU) 801, read-only memory (ROM) 802 and random access memory (RAM) 803 are interconnected via bus 804.

[0424] The input / output interface 810 is also connected to the bus 804. The input unit 811, output unit 812, storage unit 813, communication unit 814 and driver 815 are connected to the input / output interface 810.

[0425] Input unit 811 includes, for example, a keyboard, mouse, microphone, touchpad, input terminal, etc. Output unit 812 includes, for example, a display, speaker, output terminal, etc. Storage unit 813 includes, for example, a hard disk, RAM disk, non-volatile memory, etc. Communication unit 814 includes, for example, a network interface. Driver 815 drives removable media 821 such as a hard disk, optical disk, magneto-optical disk, or semiconductor memory.

[0426] In a computer configured as described above, the CPU 801 loads a program stored in the storage unit 813 into the RAM 803 via, for example, the input / output interface 810 and the bus 804, and executes the program to perform the aforementioned series of processes. Furthermore, the RAM 803 appropriately stores data required by the CPU 801 to perform various types of processing.

[0427] A program to be executed by a computer can be recorded on a removable medium 821, such as a packaging medium, and applied and provided. In this case, by attaching the removable medium 821 to the drive 815, the program can be installed into the storage unit 813 via the input / output interface 810.

[0428] Furthermore, the program can be provided via wired or wireless transmission media such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0429] In addition to the methods mentioned above, the program can be pre-installed in ROM 802 or storage unit 813.

[0430] <Applications of this technology>

[0431] This technique can be applied to any image encoding / decoding method. That is, the specifications of various types of processing related to image encoding / decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and not limited to the examples above, as long as they do not contradict the technique described above. Furthermore, processing parts can be omitted as long as they do not contradict the technique described above.

[0432] Furthermore, this technology can be applied to multi-view image encoding / decoding systems that perform encoding / decoding of images comprising multiple viewpoints (viewpoints). In this case, this technology is simply applied to the encoding / decoding of each viewpoint (viewpoint).

[0433] Furthermore, this technology can be applied to layered image encoding (scalable encoding) / decoding systems that encode / decode multi-layered (layered) images to achieve scalability for predetermined parameters. In this case, the technology is simply applied to the encoding / decoding of each layer.

[0434] Furthermore, in the above description, the encoding device 100, the decoding device 200, the image encoding device 300, and the image decoding device 400 have been described as application examples of this technology, but this technology can be applied to any configuration.

[0435] For example, this technology can be applied to various electronic devices, such as transmitters and receivers in satellite broadcasting (e.g., television receivers and mobile phones), cable broadcasting such as cable television, distribution over the Internet, and distribution to terminals via cellular communications, or devices for recording images on and reproducing images from media such as optical discs, magnetic disks, and flash memory (e.g., hard disk recorders and camera devices).

[0436] Furthermore, this technology can be implemented as a configuration as part of a device, such as a processor (e.g., a video processor) in a system-wide integrated system (LSI), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which additional functions are added to the unit (e.g., a video set) (i.e., a configuration as part of a device).

[0437] Furthermore, this technology can also be applied to network systems comprising multiple devices. For example, this technology can be implemented as cloud computing where multiple devices collaborate, share, and process data via a network. For example, this technology can be implemented as a cloud service that provides image (moving image) services to any terminal such as a computer, audiovisual (AV) device, portable information processing terminal, or Internet of Things (IoT) device.

[0438] Note that in this specification, the term "system" refers to a group of multiple configuration elements (devices, modules (parts), etc.), and it is irrelevant whether all configuration elements are housed in the same enclosure. Therefore, multiple devices housed in different enclosures and connected via a network, as well as a single device housing multiple modules in one enclosure, are both systems.

[0439] <Fields and Applications Where Existing Technology Is Applicable>

[0440] Systems, devices, and processing units utilizing this technology can be used in any field, such as transportation, healthcare, crime prevention, agriculture, animal husbandry, mining, beauty, factories, home appliances, weather, and nature monitoring. Furthermore, the application in any field is also arbitrary.

[0441] For example, this technology can be applied to systems and devices provided for providing content for enjoyment, etc. Furthermore, for example, this technology can also be applied to traffic systems and devices such as traffic condition monitoring and automatic driving control. Furthermore, for example, this technology can also be applied to systems and devices provided for safety. Furthermore, for example, this technology can be applied to systems and devices provided for the automatic control of machines, etc. Furthermore, for example, this technology can also be applied to systems and devices provided for agriculture or animal husbandry. Furthermore, this technology can also be applied to systems and devices for monitoring the natural state of things such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, this technology can also be applied to systems and devices provided for sports.

[0442] <Other>

[0443] Note that the term "flag" in this specification refers to information used to identify multiple states, including not only information identifying the states of true (1) and false (0), but also information capable of identifying three or more states. Therefore, the value of a "flag" can be, for example, the binary value 1 / 0, or a ternary value or more. That is, the number of bits constituting a "flag" is arbitrary and can be one or more bits. Furthermore, it is assumed that the identification information (including flags) is not only included in the bitstream in the form of identification information, but also in the bitstream in the form of information showing the difference between the identification information and specific reference information. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also the information showing the difference relative to the reference information.

[0444] Furthermore, various types of information (metadata, etc.) about the encoded data (bitstream) can be sent or recorded in any form, as long as the various types of information are associated with the encoded data. Here, the term "associated" means, for example, that one piece of data can be used (linked) while processing another piece of data. That is, data that are associated with each other can be collected as one piece of data or can be separate pieces of data. For example, information associated with the encoded data (image) can be sent on a transmission path different from the transmission path of the encoded data (image). Furthermore, for example, information associated with the encoded data (image) can be recorded on a different recording medium (or another recording area of ​​the same recording medium) than the encoded data (image). Note that this "association" can be a part of the data rather than the entire data. For example, an image and the information corresponding to the image can be associated with each other in any unit such as multiple frames, a single frame, or a portion of a frame.

[0445] Note that in this specification, terms such as “combine,” “reuse,” “add,” “integrate,” “include,” “store,” and “insert” mean putting multiple things into one thing, such as putting coded data and metadata into one data, and also mean one method of “association” mentioned above.

[0446] Furthermore, the implementation of this technology is not limited to the above-described implementation, and various modifications can be made without departing from the spirit of this technology.

[0447] For example, a configuration described as one device (or processing unit) can be divided and configured into multiple devices (or processing units). Conversely, configurations described as multiple devices (or processing units) can be collectively configured into one device (or processing unit). Furthermore, configurations other than those described above can be added to the configuration of each device (or each processing unit). Additionally, a portion of the configuration of a particular device (or processing unit) can be included in the configuration of another device (or another processing unit), provided that the system configuration and operation are substantially the same as a whole.

[0448] Furthermore, for example, the above procedure can be executed by any device. In this case, the device only needs to have the necessary functions (function blocks, etc.) and obtain the necessary information.

[0449] Furthermore, for example, each step of a flowchart can be executed by a single device, or it can be shared and executed by multiple devices. Additionally, when a step includes multiple processes, these processes can be executed by a single device or shared and executed by multiple devices. In other words, multiple processes included in a step can be executed as processes of multiple steps. Conversely, processes described as multiple steps can be executed together as a single step.

[0450] Furthermore, for example, in a program executed by a computer, the processing of the steps describing the program can be executed chronologically according to the order described in this specification, or it can be executed in parallel or individually at necessary timings when called. That is, as long as no contradiction occurs, the processing of each step can be executed in an order different from the above order. In addition, the processing of the steps describing the program can be executed in parallel with the processing of another program, or it can be combined with the processing of another program.

[0451] Furthermore, for example, multiple technologies related to this technology can be implemented independently as a single entity, as long as there is no contradiction. Of course, any number of these technologies can be implemented together. For example, part or all of the technology described in any embodiment can be combined with part or all of the technology described in another embodiment. Furthermore, any part or all of the aforementioned technologies can be combined with another technology not described above.

[0452] Note that this technology can also have the following configurations.

[0453] (1) An image processing apparatus, comprising:

[0454] Encoding unit, the encoding unit being configured to:

[0455] If the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of the secondary transform control information, which is control information related to the secondary transform, is omitted; and

[0456] If the transform skip flag of all components in the valid components of the image encoding is false, the secondary transform control information is encoded.

[0457] (2) The image processing apparatus according to (1), wherein,

[0458] When the tree type for image encoding is a single tree, the encoding unit performs the following operations:

[0459] If at least one of the following is true: the Y-component transform skip flag (as a transform skip flag for the Y component), the Cb-component transform skip flag (as a transform skip flag for the Cb component), or the Cr-component transform skip flag (as a transform skip flag for the Cr component), the encoding of the secondary transform control information is omitted.

[0460] When the Y component transformation skip flag, the Cb component transformation skip flag, and the Cr component transformation skip flag are false, the secondary transformation control information is encoded.

[0461] (3) The image processing apparatus according to (1) or (2), wherein,

[0462] When the image encoding tree type is dual-tree luminance, the encoding unit performs the following operations:

[0463] When the Y-component transform skip flag, which serves as the transform skip flag for the Y-component, is true, the encoding of the secondary transform control information is omitted, and

[0464] If the Y-component transform skip flag is false, the secondary transform control information is encoded.

[0465] (4) The image processing apparatus according to any one of (1) to (3), wherein,

[0466] When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations:

[0467] If at least one of the Cb component transform skip flag (as a Cb component transform skip flag) or the Cr component transform skip flag (as a Cr component transform skip flag) is true, the encoding of the secondary transform control information is omitted, and

[0468] When the Cb component transformation skip flag and the Cr component transformation skip flag are false, the secondary transformation control information is encoded.

[0469] (5) The image processing apparatus according to any one of (1) to (4), wherein,

[0470] The secondary transformation control information includes a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation.

[0471] (6) The image processing apparatus according to any one of (1) to (5), wherein,

[0472] The second transformation is the low-frequency non-separable transformation (LFNST).

[0473] (7) The image processing apparatus according to any one of (1) to (6), wherein,

[0474] The encoding unit performs the following operations:

[0475] If the transform skip flag for all valid components in image coding is true, the encoding of the secondary transform control information is omitted, and

[0476] The secondary transformation control information is encoded when the transform skip flag of at least one of the valid components in the image encoding is false.

[0477] (8) The image processing apparatus according to any one of (1) to (7), wherein,

[0478] When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations:

[0479] If the transform skip flag for all valid components in the dual-tree chromaticity is true, the encoding of the secondary transform control information is omitted, and

[0480] If the transform skip flag of at least one of the effective components in the dual-tree chromaticity is false, the secondary transform control information is encoded.

[0481] (9) The image processing apparatus according to any one of (1) to (8), wherein,

[0482] When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations:

[0483] If the transform skip flag of at least one of the effective components in the dual-tree chromaticity is true, the encoding of the secondary transform control information is omitted, and

[0484] If the transform skip flag of all components in the effective components of the dual-tree chromaticity is false, the secondary transform control information is encoded.

[0485] (10) An image processing method, comprising:

[0486] When the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of secondary transform control information, which is control information related to the secondary transform, is omitted; and

[0487] If the transform skip flag of all components in the valid components of the image encoding is false, the secondary transform control information is encoded.

[0488] (11) An image processing apparatus, comprising:

[0489] Decoding unit, the decoding unit being configured to:

[0490] If the transform skip flag of at least one of the valid components in the image decoding is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, wherein the secondary transform control information is control information related to the secondary transform; and

[0491] If the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0492] (12) The image processing apparatus according to (11), wherein,

[0493] When the tree type for image decoding is a single tree, the decoding unit performs the following operations:

[0494] If at least one of the Y-component transform skip flag (as a transform skip flag for the Y-component), the Cb-component transform skip flag (as a transform skip flag for the Cb-component), or the Cr-component transform skip flag (as a transform skip flag for the Cr-component) is true, the decoding of the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0495] If the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information.

[0496] (13) The image processing apparatus according to (11) or (12), wherein,

[0497] When the image decoding tree type is dual-tree luminance, the decoding unit performs the following operations:

[0498] If the Y-component transform skip flag, which serves as a transform skip flag for the Y-component, is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0499] If the Y component transform skip flag is false, the bit stream is decoded to generate the secondary transform control information.

[0500] (14) The image processing apparatus according to any one of (11) to (13), wherein,

[0501] When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations:

[0502] If at least one of the Cb component transform skip flag (as a Cb component transform skip flag) or the Cr component transform skip flag (as a Cr component transform skip flag) is true, the decoding of the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0503] If the Cb component transform skip flag and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information.

[0504] (15) The image processing apparatus according to any one of (11) to (14), wherein,

[0505] The secondary transformation control information includes a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation.

[0506] (16) The image processing apparatus according to any one of (11) to (15), wherein,

[0507] The second transformation is the low-frequency non-separable transformation (LFNST).

[0508] (17) The image processing apparatus according to any one of (11) to (16), wherein,

[0509] The decoding unit performs the following operations:

[0510] If the transform skip flag is true for all valid components in the image decoding, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0511] If the transform skip flag of at least one of the valid components in the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0512] (18) The image processing apparatus according to any one of (11) to (17), wherein,

[0513] When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations:

[0514] If the transform skip flag for all valid components in the dual-tree chroma key is true, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0515] If the transform skip flag of at least one of the valid components in the dual-tree chroma is false, the bitstream is decoded to generate the secondary transform control information.

[0516] (19) The image processing apparatus according to any one of (11) to (18), wherein,

[0517] When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations:

[0518] If the transform skip flag of at least one of the valid components in the dual-tree chroma key is true, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform.

[0519] If the transform skip flag of all components in the valid components of the dual-tree chroma is false, the bitstream is decoded to generate the secondary transform control information.

[0520] (20) An image processing method, comprising:

[0521] If the transform skip flag of at least one of the valid components in the image decoding is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, wherein the secondary transform control information is control information related to the secondary transform; and

[0522] If the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

[0523] List of reference numerals

[0524] 100 encoding device

[0525] 101 LFNST Identifier Encoding Unit

[0526] 200 Decoding Device

[0527] 201 LFNST Identifier Decoding Unit

[0528] 300 Image Encoding Device

[0529] 301 Control Unit

[0530] 313 Orthogonal Transformation Unit

[0531] 315 coding unit

[0532] 400 Image Decoding Device

[0533] 412 Decoding Units

[0534] 414 Inverse Orthogonal Transformation Unit

Claims

1. An image processing apparatus, comprising: Encoding unit, the encoding unit being configured to: If the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of the secondary transform control information, which is control information related to the secondary transform, is omitted. and If the transform skip flag of all components in the valid components of the image encoding is false, the secondary transform control information is encoded.

2. The image processing apparatus according to claim 1, wherein, When the tree type for image encoding is a single tree, the encoding unit performs the following operations: If at least one of the following is true: the Y-component transform skip flag (as a transform skip flag for the Y component), the Cb-component transform skip flag (as a transform skip flag for the Cb component), or the Cr-component transform skip flag (as a transform skip flag for the Cr component), the encoding of the secondary transform control information is omitted. When the Y component transformation skip flag, the Cb component transformation skip flag, and the Cr component transformation skip flag are false, the secondary transformation control information is encoded.

3. The image processing apparatus according to claim 1, wherein, When the image encoding tree type is dual-tree luminance, the encoding unit performs the following operations: When the Y-component transform skip flag, which serves as the transform skip flag for the Y-component, is true, the encoding of the secondary transform control information is omitted, and If the Y-component transform skip flag is false, the secondary transform control information is encoded.

4. The image processing apparatus according to claim 1, wherein, When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations: If at least one of the Cb component transform skip flag (as a Cb component transform skip flag) or the Cr component transform skip flag (as a Cr component transform skip flag) is true, the encoding of the secondary transform control information is omitted, and When the Cb component transformation skip flag and the Cr component transformation skip flag are false, the secondary transformation control information is encoded.

5. The image processing apparatus according to claim 1, wherein, The secondary transformation control information includes a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation.

6. The image processing apparatus according to claim 1, wherein, The second transformation is a low-frequency non-separable transformation (LFNST).

7. The image processing apparatus according to claim 1, wherein, The encoding unit performs the following operations: If the transform skip flag for all valid components in image coding is true, the encoding of the secondary transform control information is omitted, and The secondary transformation control information is encoded when the transform skip flag of at least one of the valid components in the image encoding is false.

8. The image processing apparatus according to claim 1, wherein, When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations: If the transform skip flag for all valid components in the dual-tree chromaticity is true, the encoding of the secondary transform control information is omitted, and If the transform skip flag of at least one of the effective components in the dual-tree chromaticity is false, the secondary transform control information is encoded.

9. The image processing apparatus according to claim 1, wherein, When the image encoding tree type is dual-tree chroma, the encoding unit performs the following operations: If the transform skip flag of at least one of the effective components in the dual-tree chromaticity is true, the encoding of the secondary transform control information is omitted, and If the transform skip flag of all components in the effective components of the dual-tree chromaticity is false, the secondary transform control information is encoded.

10. An image processing method, comprising: If the transform skip flag of at least one of the valid components in the image encoding is true, the encoding of the secondary transform control information, which is control information related to the secondary transform, is omitted. as well as If the transform skip flag of all components in the valid components of the image encoding is false, the secondary transform control information is encoded.

11. An image processing apparatus, comprising: Decoding unit, the decoding unit being configured to: If the transform skip flag of at least one of the valid components in the image decoding is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, wherein the secondary transform control information is control information related to the secondary transform; and If the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

12. The image processing apparatus according to claim 11, wherein, When the tree type for image decoding is a single tree, the decoding unit performs the following operations: If at least one of the Y-component transform skip flag (as a transform skip flag for the Y-component), the Cb-component transform skip flag (as a transform skip flag for the Cb-component), or the Cr-component transform skip flag (as a transform skip flag for the Cr-component) is true, the decoding of the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the Y component transform skip flag, the Cb component transform skip flag, and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information.

13. The image processing apparatus according to claim 11, wherein, When the image decoding tree type is dual-tree luminance, the decoding unit performs the following operations: If the Y-component transform skip flag, which serves as a transform skip flag for the Y-component, is true, decoding related to the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the Y component transform skip flag is false, the bit stream is decoded to generate the secondary transform control information.

14. The image processing apparatus according to claim 11, wherein, When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations: If at least one of the Cb component transform skip flag (as a Cb component transform skip flag) or the Cr component transform skip flag (as a Cr component transform skip flag) is true, the decoding of the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the Cb component transform skip flag and the Cr component transform skip flag are false, the bit stream is decoded to generate the secondary transform control information.

15. The image processing apparatus according to claim 11, wherein, The secondary transformation control information includes a secondary transformation identifier, which is an identifier indicating the type of the secondary transformation.

16. The image processing apparatus according to claim 11, wherein, The second transformation is a low-frequency non-separable transformation (LFNST).

17. The image processing apparatus according to claim 11, wherein, The decoding unit performs the following operations: If the transform skip flag is true for all valid components in the image decoding, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of at least one of the valid components in the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

18. The image processing apparatus according to claim 11, wherein, When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations: If the transform skip flag for all valid components in the dual-tree chroma key is true, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of at least one of the valid components in the dual-tree chroma is false, the bitstream is decoded to generate the secondary transform control information.

19. The image processing apparatus according to claim 11, wherein, When the image decoding tree type is dual-tree chroma, the decoding unit performs the following operations: If the transform skip flag of at least one of the valid components in the dual-tree chroma key is true, decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform. If the transform skip flag of all components in the valid components of the dual-tree chroma is false, the bitstream is decoded to generate the secondary transform control information.

20. An image processing method, comprising: If the transform skip flag of at least one of the valid components in the image decoding is true, the decoding associated with the secondary transform control information is omitted, and the value of the secondary transform control information is estimated to be a predetermined value indicating the skipping of the secondary transform, the secondary transform control information being control information associated with the secondary transform; as well as If the transform skip flag of all components in the valid components of the image decoding is false, the bitstream is decoded to generate the secondary transform control information.

Images