Non-transitory computer-readable medium and transmission method

Abstract

To suppress reduction in encoding efficiency.SOLUTION: In encoding of an image, a transform skip flag that is flag information indicative of whether or not a transform process for transforming a residual between the image and a prediction image of the image into coefficient data is to be skipped for each component is generated, the generated transform skip flag is encoded to generate encoded data of the transform skip flag, and a bit stream including the encoded data of the generated transform skip flag is generated. The present disclosure can be applied to, for example, an image processing apparatus, an image encoding apparatus, an image decoding apparatus, a transmission apparatus, a reception apparatus, a transmission / reception apparatus, an information processing apparatus, an imaging apparatus, a reproduction apparatus, a bit stream generation method, a coefficient data generation method, a quantization coefficient generation method, or the like.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a non-transitory computer-readable medium, and more particularly to a non-transitory computer-readable medium that can suppress a reduction in coding efficiency. [Background technology]

[0002] BACKGROUND ART Conventionally, in image coding, a method has been proposed in which coding is performed by skipping (omitting) a conversion process that converts the residual between an image and its predicted image into coefficient data (for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, "Versatile Video Coding (Draft 5)", JVET-N1001-v10, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 14th Meeting: Geneva, CH, 19-27 Mar. 2019 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the method described in Non-Patent Document 1, this transform skip mode is applied only to the luminance component (luminance transform skip). In other words, this transform skip mode cannot be applied to the chrominance component. This may result in a decrease in coding efficiency.

[0005] The present disclosure has been made in light of such circumstances, and makes it possible to suppress a decrease in coding efficiency. [Means for solving the problem]

[0006] A non-transitory computer-readable medium according to one aspect of the present technology is a non-transitory computer-readable medium that stores a bitstream configured to be processed by one or more processing devices, and includes encoded data of a transform skip flag, which is flag information indicating, for each component, whether or not to skip a transform process that converts a residual between an image and a predicted image of the image into coefficient data when encoding the image.

[0007] In a non-transitory computer-readable medium according to one aspect of the present technology, a bitstream configured to be processed by one or more processing devices is stored, the bitstream including encoded data of a transform skip flag, which is flag information indicating, for each component, whether or not to skip a transform process that converts a residual between an image and a predicted image of that image into coefficient data when encoding the image. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 10 is a diagram illustrating an extension of a conversion skip. [Figure 2] FIG. 1 is a block diagram illustrating an example of the main configuration of an image encoding device. [Figure 3] FIG. 2 is a block diagram illustrating an example of the main configuration of an encoding unit. [Figure 4] 10 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 5] 10 is a flowchart illustrating an example of the flow of an encoding process. [Figure 6] FIG. 1 is a block diagram illustrating an example of the main configuration of an image decoding device. [Figure 7] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding unit. [Figure 8] 10 is a flowchart showing an example of the flow of an image decoding process. [Figure 9] 10 is a flowchart illustrating an example of the flow of a decoding process. [Figure 10]FIG. 10 is a diagram illustrating an example of the syntax of a TU. [Figure 11] FIG. 10 is a diagram illustrating an example of the syntax of conversion mode information. [Figure 12] FIG. 10 is a diagram illustrating an example of the syntax of a TS residual coding mode. [Figure 13] FIG. 10 is a diagram illustrating an example of the semantics of conversion mode information. [Figure 14] FIG. 10 is a diagram illustrating another example of the syntax of the conversion mode information. [Figure 15] FIG. 10 is a diagram illustrating an example of the syntax of a sequence parameter set. [Figure 16] FIG. 10 is a diagram illustrating an example of the syntax of a TU. [Figure 17] FIG. 10 illustrates an example of the semantics of a sequence parameter set. [Figure 18] FIG. 10 is a diagram illustrating an example of the syntax of a sequence parameter set. [Figure 19] FIG. 10 is a diagram illustrating an example of the syntax of a TU. [Figure 20] FIG. 10 illustrates an example of the semantics of a sequence parameter set. [Figure 21] FIG. 2 is a block diagram illustrating an example of the main configuration of a quantization unit. [Figure 22] 10 is a flowchart illustrating an example of the flow of a quantization process. [Figure 23] FIG. 2 is a block diagram illustrating an example of the main configuration of an inverse quantization unit. [Figure 24] 10 is a flowchart illustrating an example of the flow of an inverse quantization process. [Figure 25] FIG. 10 is a diagram illustrating an example of the syntax of a quantization parameter. [Figure 26] FIG. 10 is a diagram illustrating an example of the syntax of a context variable. [Figure 27] FIG. 10 is a diagram illustrating an example of the syntax of a context variable. [Figure 28] FIG. 10 is a diagram illustrating switching of sign encoding modes. [Figure 29] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described in the following order. 1. Literature supporting technical content and technical terminology 2. Skip conversion 3. First embodiment (extension of conversion skip) 4. Second embodiment (quantization parameter correction) 5. Third embodiment (sharing of context variables) 6. Fourth embodiment (encoding / decoding mode control of sign code) 7. Supplementary Notes

[0010] <1. Literature supporting technical content and technical terminology> The scope of disclosure of the present technology includes not only the contents described in the examples but also the contents described in the following non-patent documents that were publicly known at the time of filing.

[0011] Non-patent document 1: (mentioned above) Non-patent document 2: Takeshi Tsukuba, Masaru Ikeda, Yoichi Yagasaki, Teruhiko Suzuki, "CE8: Chroma Transform Skip (CE8-3.2)", JVET-O0081-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 15th Meeting: Gothenburg, SE, 3-12 July 2019 Non-Patent Document 3: Tung Nguyen, Benjamin Bross, Heiko Schwarz, Detlev Marpe, Thomas Wiegand, "Non-CE8: Minimum Allowed QP for Transform Skip Mode", JVET-O0405-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 15th Meeting: Gothenburg, SE, 3-12 July 2019 Non-Patent Document 4: Jianle Chen, Yan Ye, Seung Hwan Kim, "Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)", JVET-O2002-v2, Joint Video Experts Team (JVET), of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 15th Meeting: Gothenburg, SE, 3-12 July 2019 Non-Patent Document 5: Takeshi Tsukuba, Masaru Ikeda, Yoichi Yagasaki, Teruhiko Suzuki, "CE8-2.1: Transform Skip for Chroma with limiting maximum number of context-coded bin in TS residual coding", JVET-P0058-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Geneva, CH, 1-11 October 2019 Non-patent document 6: Gordon Clare, Felix Henry, Takeshi Tsukuba, Masaru Ikeda, Yoich Yagasaki, Teruhiko Suzuki, "CE8-4.1: BDPCM and Transform skip for Chroma", JVET-P0059-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Geneva, CH, 1-11 October 2019 Non-patent document 7: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04 / 2017 Non-patent document 8: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12 / 2016

[0012] In other words, the contents of the above-mentioned non-patent documents are also used as a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and to meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and to meet the support requirements of the claims.

[0013] Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region of an image (picture) or a processing unit refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" is intended to include any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), transform block, sub-block, macroblock, tile, or slice, as described in the above-mentioned non-patent document.

[0014] Furthermore, when specifying such block sizes, the block sizes may be specified not only directly but also indirectly. For example, the block sizes may be specified using identification information for identifying the sizes. Furthermore, for example, the block sizes may be specified by the ratio or difference with respect to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. This may reduce the amount of information and improve coding efficiency. Furthermore, the specification of the block sizes may also include specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0015] Furthermore, in this specification, "encoding" refers not only to the overall process of converting an image into a bitstream, but also to some of the processes. For example, it not only includes processes that encompass prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also includes a process that collectively refers to quantization and arithmetic coding, a process that encompasses prediction processing, quantization, and arithmetic coding, etc. Similarly, "decoding" refers not only to the overall process of converting a bitstream into an image, but also to some of the processes. For example, it not only includes processes that encompass inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also includes a process that encompasses inverse arithmetic decoding and inverse quantization, a process that encompasses inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0016] <2. Skip conversion> <Skip conversion of luminance component> In the past, in coding still images and moving images, the image input to a coding device was generally a color image having a luminance component and a color component (which may include a color difference component). In such image coding, for example, as described in Non-Patent Document 1, a coding method has been devised in which the luminance component is coded by skipping (omitting) the conversion process that converts the residual between the image and its predicted image into coefficient data.

[0017] However, this transform skip mode could not be applied to color components, which could result in a decrease in coding efficiency for color components in screen content where transform skip is effective.

[0018] <Extended conversion skip flag> Therefore, the transform skip is extended so that it can be set for each component (for each luminance component or color component), and so that the transform skip can be performed not only for the luminance component but also for the color components. For example, in the image encoding / decoding method described in the above-mentioned non-patent document, the skip setting can be indicated by a transform skip flag (transform_skip_flag). Furthermore, a component can be indicated by a component identifier (cIdx). Therefore, for example, as in Method 1 in the top row of the table in Figure 1, the value of this transform skip flag is set corresponding to the value of the component identifier.

[0019] For example, an image processing device may include a flag generation unit that generates a transform skip flag, which is flag information indicating for each component whether or not to skip the transform process that converts the residual between an image and a predicted image of that image into coefficient data when encoding the image; a flag encoding unit that encodes the transform skip flag generated by the flag generation unit and generates encoded data of the transform skip flag; and a bit stream generation unit that generates a bit stream including the encoded data of the transform skip flag generated by the flag encoding unit.

[0020] Furthermore, for example, when generating a bitstream, a transform skip flag is generated, which is flag information indicating for each component whether or not to skip the transform process that converts the residual between the image and the predicted image of that image into coefficient data when encoding the image, the generated transform skip flag is encoded, encoded data of the transform skip flag is generated, and a bitstream including the encoded data of the generated transform skip flag is generated.

[0021] This allows the decoding side to be provided with a transform skip setting for each component. Therefore, when the transform skip is applied to a color component, the decoding side can correctly decode the bitstream. Therefore, it is possible to suppress a decrease in coding efficiency compared to when the transform skip is applicable only to the luminance component.

[0022] Then, the decoding side may be configured to be able to acquire this conversion skip flag.

[0023] For example, an image processing device may be provided with a flag decoding unit that decodes coded data of a transform skip flag corresponding to a component identifier and obtains the transform skip flag corresponding to the component identifier.

[0024] Furthermore, for example, when generating coefficient data, coded data of a transform skip flag corresponding to a component identifier may be decoded to obtain the transform skip flag corresponding to that component identifier.

[0025] By doing so, it is possible to apply the transform skip setting for each component indicated by the transform skip flag during decoding. Therefore, it is possible to correctly decode the bitstream of the color component to which the transform skip is applied. Therefore, it is possible to suppress a decrease in coding efficiency compared to when the transform skip is applicable only to the luminance component.

[0026] Furthermore, when encoding an image, the transform processing of color components may be controlled based on the setting of the transform skip for each component using the transform skip flag described above. That is, the transform skip may be applied to the transform processing of color components. This makes it possible to suppress a decrease in encoding efficiency compared to when the transform skip is applicable only to the luminance component.

[0027] Similarly, when decoding coded data of an image, the inverse transform process of converting coefficient data of a color component into a residual between the image and a predicted image may be controlled based on the transform skip setting for each component using the transform skip flag described above. That is, the transform skip may be applied to the inverse transform process of the color component. By doing so, it is possible to correctly decode the bit stream of the color component to which the transform skip has been applied. Therefore, it is possible to suppress a decrease in coding efficiency compared to when the transform skip is applicable only to the luminance component.

[0028] As described above, by setting the transform skip for each component, it is possible to set the transform skip for the color component independently of the luminance component. Therefore, it is possible to set the transform skip for the color component depending on whether the transform skip is enabled for the color component. In this way, it is possible to suppress a decrease in coding efficiency compared to when the transform skip setting for the luminance component is applied to the color component.

[0029] <Encoding mode control> Furthermore, in the method described in Non-Patent Document 1, the encoding mode (decoding mode) of the coefficient data of the luminance component was controlled depending on whether or not a transform skip was performed. For example, when a transform skip was performed, the coefficient data of the luminance component was encoded in a TS residual encoding mode optimized for the coefficient data that was transformed skipped (the encoded data of the coefficient data of the luminance component was decoded in a TS residual decoding mode optimized for the coefficient data that was transformed skipped). In contrast, when a transform skip was not performed, the coefficient data of the luminance component was encoded in a non-TS residual encoding mode optimized for the coefficient data that was transformed (the encoded data of the coefficient data of the luminance component was decoded in a non-TS residual decoding mode optimized for the coefficient data that was transformed).

[0030] However, since the transform skip is not applied to the chrominance components, such control of the encoding mode (decoding mode) is not performed either. Therefore, when the transform skip is applied to the chrominance components as described above, there is a risk that the encoding efficiency will be reduced compared to when such control of the encoding mode (decoding mode) is performed.

[0031] Therefore, such control of the encoding mode (decoding mode) may also be applied when applying transform skip to color components as in the above-mentioned method 1. For example, as in method 1-1 in the second row from the top of the table in Fig. 1, for each value of the component identifier, when transform skip is applied, the TS residual encoding mode (TS residual decoding mode) may be applied as the encoding mode (decoding mode), and when transform skip is not applied (in the case of no transform skip), the non-TS residual encoding mode (non-TS residual decoding mode) may be applied as the encoding mode (decoding mode).

[0032] For example, based on a transform skip flag corresponding to a component identifier, the coding mode of the coefficient data corresponding to that component identifier may be controlled to either a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process, and the coefficient data corresponding to the component identifier may be coded using the coding mode set in this way to generate coded data of the coefficient data.

[0033] Furthermore, for example, based on a transform skip flag corresponding to a component identifier, the decoding mode of the coded data of coefficient data corresponding to that component identifier may be controlled to either a TS residual decoding mode, which is a mode for skipping the inverse transform process of converting coefficient data into residuals between an image and a predicted image, or a non-TS residual decoding mode, which is a mode for not skipping the inverse transform process, and the coded data of coefficient data corresponding to the component identifier may be decoded using the decoding mode set in this way to generate coefficient data corresponding to that component identifier.

[0034] By doing this, it is possible to apply an encoding mode (decoding mode) according to the setting of the transform skip, thereby suppressing a decrease in encoding efficiency compared to when encoding (decoding) is performed using a single encoding mode (decoding mode).

[0035] <Transform skip residual coding use flag> A flag indicating selection between the TS residual coding mode (TS residual decoding mode) and the non-TS residual coding mode (non-TS residual decoding mode) may be applied. For example, as in method 1-1-1 in the third row from the top of the table in Fig. 1, a residual coding mode selection flag (also referred to as a transform skip residual coding use flag) indicating the selection setting of the coding mode may be applied.

[0036] For example, when encoding, a transform skip residual coding usage flag may be generated, which is flag information indicating whether to apply a TS residual coding mode or a non-TS residual coding mode, and the transform skip residual coding usage flag may be encoded to generate a bitstream including encoded data of the transform skip residual coding usage flag.

[0037] Furthermore, during decoding, the encoded data of the transform skip residual coding use flag may be decoded, a transform skip residual coding use flag corresponding to the component identifier may be generated, and based on the transform skip residual coding use flag corresponding to the component identifier, the decoding mode of the encoded data of the coefficient data corresponding to the component identifier may be controlled to be the TS residual decoding mode or the non-TS residual decoding mode.

[0038] For example, such a transform skip residual coding use flag may be set at a high level such as a sequence parameter. By doing so, even an encoder or decoder that does not support transform skip can correctly perform encoding and decoding based on this flag. Therefore, it is possible to suppress a decrease in coding efficiency. In other words, it is possible to omit the implementation of the TS residual coding mode (TS residual decoding mode) in the encoder or decoder, thereby suppressing an increase in circuit size.

[0039] <Conversion skip residual coding use specific mode flag> A flag indicating whether to apply a TS residual coding mode (TS residual decoding mode) or a non-TS residual coding mode (non-TS residual decoding mode) in a specific mode may be applied. For example, a flag (also referred to as a transform skip residual coding use specific mode flag) that enables selection of a coding mode in a specific mode may be applied, as in method 1-1-2 in the fourth row from the top of the table in Fig. 1 .

[0040] For example, when encoding, a transform skip residual coding use specific mode flag may be generated, which is flag information indicating whether to apply a TS residual coding mode or a non-TS residual coding mode in a specific mode, and the transform skip residual coding use specific mode flag may be encoded to generate a bitstream including encoded data of the transform skip residual coding use specific mode flag.

[0041] Furthermore, during decoding, the encoded data of the transform skip residual coding use specific mode flag may be decoded, a transform skip residual coding use specific mode flag corresponding to the component identifier may be generated, and based on the transform skip residual coding use specific mode flag corresponding to the component identifier, the decoding mode of the encoded data of the coefficient data corresponding to the component identifier may be controlled to be TS residual decoding mode or non-TS residual decoding mode.

[0042] For example, such a transform skip residual coding use specific mode flag may be set at a high level in a sequence parameter, etc. In this way, it becomes possible to switch between a TS residual decoding mode and a non-TS residual decoding mode in a specific mode.

[0043] The above methods 1, 1-1, 1-1-1, and 1-1-2 will be described later in the first embodiment.

[0044] <Quantization parameter correction> In Non-Patent Document 3, when a quantization parameter QP<4 is applied to a luminance transformation block to which a luminance transformation skip is applied, there is a risk of a decrease in PSNR (Peak Signal-to-Noise Ratio). As a countermeasure, a method is proposed in which the quantization parameter QP applied to the luminance transformation block in the case of a luminance transformation skip is clipped to QP=4 or QP=QpPrimeTsMin (minimum TSQP).

[0045] Similarly, when transform skip is applied to color components, there is a risk of a decrease in PSNR. However, transform skip has not been applied to color components in the past, and naturally, this point has not been taken into consideration.

[0046] Therefore, for example, as in method 1-2 in the fifth row from the top of the table in Fig. 1, for each value of the component identifier, in the case of transform skip, the quantization parameter may be corrected. In other words, when transform skip is applied, the quantization parameter may be corrected. Furthermore, such control may be performed for each component.

[0047] For example, in an image processing device, when encoding an image, if a transform skip flag corresponding to a component identifier indicates a transform skip, the image processing device may be provided with a quantization parameter correction unit that corrects a quantization parameter to be applied to a transform block to be processed that corresponds to the component identifier, and a quantization unit that quantizes the transform block to be processed that corresponds to the component identifier using the quantization parameter corrected by the quantization parameter correction unit.

[0048] For example, in a quantization coefficient generation method, when encoding an image, if a transform skip flag corresponding to a component identifier indicates a transform skip, the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier may be corrected, and the corrected quantization parameter may be used to quantize the transform block to be processed corresponding to that component identifier, thereby generating a quantization coefficient corresponding to that component identifier.

[0049] For example, in an image processing device, when a transform skip flag corresponding to a component identifier indicates a transform skip, the image processing device may be provided with a quantization parameter correction unit that corrects a quantization parameter to be applied to a transform block to be processed that corresponds to the component identifier, and an inverse quantization unit that uses the quantization parameter corrected by the quantization parameter correction unit to perform inverse quantization on the transform block to be processed that corresponds to the component identifier.

[0050] For example, in a coefficient data generation method, if a transform skip flag corresponding to a component identifier indicates a transform skip, the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier may be corrected, and the corrected quantization parameter may be used to perform inverse quantization on the transform block to be processed corresponding to that component identifier, thereby generating coefficient data corresponding to that component identifier.

[0051] By doing so, it is possible to suppress a decrease in PSNR in the encoder or decoder, or in both.

[0052] For example, in encoding, if a transform skip flag corresponding to a component identifier indicates a transform skip, the larger of the minimum quantization parameter of that transform skip and the quantization parameter corresponding to that component identifier may be set as the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier, and if the transform skip flag corresponding to that component identifier indicates a non-transform skip that does not perform a transform skip, the quantization parameter corresponding to that component identifier may be set as the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier.

[0053] Also, for example, in decoding, if a transform skip flag corresponding to a component identifier indicates a transform skip, the larger of the minimum quantization parameter of that transform skip and the quantization parameter corresponding to that component identifier may be set as the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier, and if the transform skip flag corresponding to that component identifier indicates a non-transform skip that does not perform a transform skip, the quantization parameter corresponding to that component identifier may be set as the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier.

[0054] By doing so, it is possible to suppress a decrease in PSNR in the encoder or decoder, or in both.

[0055] For example, when decoding, encoded data of a transform skip flag corresponding to a component identifier is decoded, and if the transform skip flag corresponding to that component identifier indicates a transform skip, the quantization parameter to be applied to the transform block to be processed corresponding to that component identifier may be corrected.

[0056] In this way, it is possible to control the correction of the quantization parameter based on the transform skip flag corresponding to the coded component identifier.

[0057] The above method 1-2 will be described later in the second embodiment.

[0058] <Sharing context variables> Furthermore, when a transform skip is introduced for a color component as described above and the transform block of that color component to which the transform skip is applied is coded using the TS residual coding mode, if independent context variables are used for the luminance component and the color component, it is necessary to add a new context variable for the color component, which increases the memory capacity for storing the context variables and may increase the hardware cost.

[0059] Therefore, in the case of a conversion skip, for example, as in Method 1-3 in the sixth row from the top of the table in Figure 1, the context variables corresponding to each binIdx in the bin string of each syntax may be shared between the luminance component and the color component.

[0060] For example, when the transform process is skipped during encoding, the context variables may be shared between the encoding of the luminance component of the coefficient data and the encoding of the chrominance component.

[0061] Furthermore, for example, when inverse transform processing is skipped during decoding, the context variables may be shared between the decoding of coded data of the luminance component of coefficient data and the decoding of coded data of the chrominance component.

[0062] This makes it possible to suppress an increase in memory size for storing context variables, thereby suppressing an increase in hardware costs.

[0063] The above methods 1 to 3 will be described later in the third embodiment.

[0064] <Sign code encoding / decoding mode control> Also, for example, as in method 1-4 in the seventh row from the top of the table in FIG. 1, the encoding / decoding method of the sign code may be switched depending on the transform skip flag corresponding to the component identifier.

[0065] For example, as in method 1-4-1 in the eighth row from the top of the table in FIG. 1, when no transform skip is used, bypass encoding / decoding may be applied to encoding / decoding of sign codes.

[0066] Also, for example, in the case of transform skip, as in method 1-4-2 in the ninth row from the top of the table in Figure 1, if the number of remaining context coding bins is equal to or greater than a threshold, context coding / decoding may be applied to the coding / decoding of the sign code, and otherwise bypass coding / decoding may be applied to the coding / decoding of the sign code.

[0067] By doing so, it is possible to suppress a decrease in coding efficiency.

[0068] The above methods 1-4, 1-4-1, and 1-4-2 will be described later in the fourth embodiment.

[0069] 3. First Embodiment <3-1. Image encoding device> The present technology described above can be applied to any apparatus, device, system, etc. For example, the present technology described above can be applied to an image encoding apparatus that encodes image data.

[0070] Fig. 2 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 100 shown in Fig. 2 is a device that encodes image data of a moving image. For example, the image encoding device 100 implements a technology described in at least one of the non-patent documents mentioned above, and encodes image data of a moving image using a method that complies with a standard described in one of those documents.

[0071] Note that Fig. 2 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 2. In other words, in the image encoding device 100, there may be processing units that are not shown as blocks in Fig. 2, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 2. This is also true for other figures that explain processing units, etc. within the image encoding device 100.

[0072] 2, the image coding device 100 includes a control unit 101, a rearrangement buffer 111, a calculation unit 112, an orthogonal transform unit 113, a quantization unit 114, a coding unit 115, and an accumulation buffer 116. The image coding device 100 also includes an inverse quantization unit 117, an inverse orthogonal transform unit 118, a calculation unit 119, an in-loop filter unit 120, a frame memory 121, a prediction unit 122, and a rate control unit 123.

[0073] <Control unit> The control unit 101 divides the video data held in the rearrangement buffer 111 into blocks (CUs, PUs, transform blocks, etc.) based on an externally or pre-specified block size of the processing unit. The control unit 101 also determines the coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization). For example, the control unit 101 can set a transform skip flag, etc.

[0074] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 101 supplies them to each block. Specifically, this is as follows.

[0075] The header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the encoding unit 115 and the prediction unit 122. The transformation information Tinfo is supplied to the encoding unit 115, the orthogonal transformation unit 113, the quantization unit 114, the inverse quantization unit 117, and the inverse orthogonal transformation unit 118. The filter information Finfo is supplied to the in-loop filter unit 120.

[0076] <Sorting buffer> Each field (input image) of video data is input to the image coding device 100 in its playback order (display order). The reordering buffer 111 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 101, the reordering buffer 111 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 111 supplies each processed input image to the calculation unit 112.

[0077] <Arithmetic section> The calculation unit 112 subtracts the predicted image P supplied from the prediction unit 122 from the image corresponding to the block of processing units supplied from the rearrangement buffer 111 to derive a prediction residual D, and supplies it to the orthogonal transformation unit 113.

[0078] <Orthogonal transformation section> The orthogonal transform unit 113 receives the prediction residual supplied from the calculation unit 112 and the transformation information Tinfo supplied from the control unit 101, and performs an orthogonal transform on the prediction residual based on the transformation information Tinfo to derive the transformation coefficient Coeff. The orthogonal transform unit 113 supplies the obtained transformation coefficient to the quantization unit 114.

[0079] <Quantization section> The quantization unit 114 receives as input the transform coefficients supplied from the orthogonal transform unit 113 and the transform information Tinfo supplied from the control unit 101, and scales (quantizes) the transform coefficients based on the transform information Tinfo. The quantization rate is controlled by the rate control unit 123. The quantization unit 114 supplies the quantized transform coefficients (also referred to as quantized transform coefficient levels) obtained by this quantization to the encoding unit 115 and the inverse quantization unit 117.

[0080] <Encoding part> The encoding unit 115 receives as input the quantization transformation coefficient levels supplied from the quantization unit 114, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 101, information about filters such as filter coefficients supplied from the in-loop filter unit 120, and information about the optimal prediction mode supplied from the prediction unit 122.

[0081] The encoding unit 115 performs entropy encoding (lossless encoding) such as CABAC (Context-based Adaptive Binary Arithmetic Code) or CAVLC (Context-based Adaptive Variable Length Code) on the quantized transform coefficient levels to generate a bit string (encoded data).

[0082] Furthermore, the encoding unit 115 derives residual information Rinfo from the quantized transform coefficient levels, encodes the residual information Rinfo, and generates a bit string.

[0083] Furthermore, the encoding unit 115 includes information about the filter supplied from the in-loop filter unit 120 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 122 in the prediction mode information Pinfo. Then, the encoding unit 115 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0084] The encoding unit 115 multiplexes the bit strings of the various types of information generated as described above to generate encoded data, and supplies the encoded data to the accumulation buffer 116.

[0085] <Accumulation buffer> The accumulation buffer 116 temporarily stores the coded data obtained by the coding unit 115. The accumulation buffer 116 outputs the stored coded data, for example, as a bit stream or the like, to the outside of the image coding device 100 at a predetermined timing. For example, this coded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, or the like. In other words, the accumulation buffer 116 also functions as a transmission unit that transmits the coded data (bit stream).

[0086] <Inverse quantization section> The inverse quantization unit 117 performs processing related to inverse quantization. For example, the inverse quantization unit 117 receives the quantized transform coefficient level Level supplied from the quantization unit 114 and the transformation information Tinfo supplied from the control unit 101 as input, and scales (inverse quantizes) the value of the quantized transform coefficient level based on the transformation information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed in the quantization unit 114. The inverse quantization unit 117 supplies the transformation coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 118. Note that the inverse quantization unit 117 is the same as the inverse quantization unit (described later) on the decoding side, and therefore the description (described later) about the decoding side can be applied to the inverse quantization unit 117.

[0087] <Inverse orthogonal transform section> The inverse orthogonal transform unit 118 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 118 receives as input the transform coefficients supplied from the inverse quantization unit 117 and the transform information Tinfo supplied from the control unit 101, and performs inverse orthogonal transform on the transform coefficients based on the transform information Tinfo to derive a prediction residual D'. Note that this inverse orthogonal transform is the inverse process of the orthogonal transform performed in the orthogonal transform unit 113. The inverse orthogonal transform unit 118 supplies the prediction residual obtained by such inverse orthogonal transform to the calculation unit 119. Note that the inverse orthogonal transform unit 118 is the same as the inverse orthogonal transform unit on the decoding side (described later), and therefore the description of the decoding side (described later) can be applied to the inverse orthogonal transform unit 118.

[0088] <Arithmetic section> The calculation unit 119 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 118 and the predicted image P supplied from the prediction unit 122. The calculation unit 119 adds the prediction residual to the predicted image corresponding to the prediction residual to derive a locally decoded image. The calculation unit 119 supplies the derived locally decoded image to the in-loop filter unit 120 and the frame memory 121.

[0089] <In-loop filter section> The in-loop filter unit 120 performs processing related to in-loop filtering. For example, the in-loop filter unit 120 receives as input a locally decoded image supplied from the calculation unit 119, filter information Finfo supplied from the control unit 101, and an input image (original image) supplied from the rearrangement buffer 111. Note that any information may be input to the in-loop filter unit 120, and information other than the above information may also be input. For example, information such as a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, and a block (CU, CTU, etc.) may be input to the in-loop filter unit 120 as needed.

[0090] The in-loop filter unit 120 performs appropriate filtering on the locally decoded image based on the filter information Finfo. The in-loop filter unit 120 also uses the input image (original image) and other input information for the filtering, as necessary.

[0091] For example, the in-loop filter unit 120 can apply four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order, as described in Non-Patent Document 1. Note that which filters to apply and in what order they are applied are arbitrary and can be selected appropriately.

[0092] Of course, the filtering process performed by the in-loop filter unit 120 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 120 may apply a Wiener filter or the like.

[0093] The in-loop filter unit 120 supplies the filtered locally decoded image to the frame memory 121. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 120 supplies the information about the filter to the encoding unit 115.

[0094] <Frame memory> The frame memory 121 performs processing related to the storage of image-related data. For example, the frame memory 121 receives as input a locally decoded image supplied from the calculation unit 119 or a locally decoded image that has been subjected to filtering processing supplied from the in-loop filter unit 120, and holds (stores) the input. The frame memory 121 also uses the locally decoded image to reconstruct a decoded image for each picture, and holds the reconstructed image (stores the reconstructed image in a buffer within the frame memory 121). The frame memory 121 supplies the decoded image (or a part thereof) to the prediction unit 122 in response to a request from the prediction unit 122.

[0095] <Prediction Department> The prediction unit 122 performs processing related to the generation of a predicted image. For example, the prediction unit 122 receives as input the prediction mode information Pinfo supplied from the control unit 101, the input image (original image) supplied from the rearrangement buffer 111, and the decoded image (or a part thereof) read from the frame memory 121. The prediction unit 122 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction by referring to the decoded image as a reference image, and performs motion compensation processing based on the prediction result to generate a predicted image. The prediction unit 122 supplies the generated predicted image to the calculation unit 112 and the calculation unit 119. Furthermore, the prediction unit 122 supplies information on the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 115 as necessary.

[0096] <Rate control section> The rate control unit 123 performs processing related to rate control. For example, the rate control unit 123 controls the rate of the quantization operation of the quantization unit 114 based on the code amount of the coded data accumulated in the accumulation buffer 116 so as to prevent overflow or underflow.

[0097] <Encoding part> Fig. 3 is a block diagram showing an example of the main configuration of the encoding unit 115 in Fig. 2. As shown in Fig. 3, the encoding unit 115 includes a transform mode information encoding unit 150, a control unit 151, a selection unit 152, a TS residual encoding unit 153, a non-TS residual encoding unit 154, and a selection unit 155.

[0098] The transform mode information encoding unit 150 performs processing related to encoding of transform mode information (transform_mode). This transform mode information is information related to the mode of transform processing by the orthogonal transform unit 113. For example, the transform mode information may include a transform skip flag (transform_skip_flag[xTbY][yTbY][cIdx]), an identifier related to primary transform (mts_idx[xTbY][yTbY][cIdx]), etc.

[0099] For example, the transform mode information encoding unit 150 may acquire transform mode information supplied from the control unit 101. The transform mode information encoding unit 150 may then encode the acquired transform mode information to generate encoded data of the transform mode information. The transform mode information encoding unit 150 may then supply the generated encoded data of the transform mode information to the accumulation buffer 116 (i.e., provide it to the decoding side).

[0100] The control unit 151 performs processing related to coding mode control. For example, the control unit 151 may acquire transform mode information (e.g., transform_skip_flag[xTbY][yTbY][cIdx]) or mts_idx[xTbY][yTbY][cIdx]) or a component identifier (cIdx) supplied from the control unit 101. Furthermore, the control unit 151 may switch the coding mode for coefficient data (quantized coefficients) between the TS residual coding mode and the non-TS residual coding mode by controlling the selection of the selection unit 152 or the selection unit 155 based on the transform skip flag corresponding to the component identifier. For example, when a transform skip is applied, the control unit 151 connects the selection unit 152 and the selection unit 155 to the TS residual coding unit 153. Furthermore, for example, when a transform skip is not applied (when a transform process is performed), the control unit 151 connects the selection unit 152 and the selection unit 155 to the non-TS residual coding unit 154.

[0101] The selection unit 152 performs processing related to selecting a supply destination of coefficient data (quantization coefficients). For example, the selection unit 152 may acquire quantization coefficients supplied from the quantization unit 114. Furthermore, under the control of the control unit 151, the selection unit 152 may supply the acquired quantization coefficients to the TS residual encoding unit 153 or the non-TS residual encoding unit 154 (whichever is specified by the control unit 151). For example, when a transform skip is applied, the selection unit 152 supplies the quantization coefficients to the TS residual encoding unit 153. Furthermore, for example, when a transform skip is not applied (when a transform process is performed), the selection unit 152 supplies the quantization coefficients to the non-TS residual encoding unit 154.

[0102] The TS residual encoder 153 performs processing related to the TS residual coding mode. For example, the TS residual encoder 153 may acquire quantized coefficients supplied from the selection unit 152. The TS residual encoder 153 may also encode the acquired quantized coefficients in the TS residual coding mode. This TS residual coding mode is a coding mode for skipping the transform process. For example, the TS residual coding mode is optimized for coefficient data for which the transform process has been skipped, such as by scanning the coefficient data from the DC component toward the high-frequency component, or by not transmitting position information of the last coefficient to the decoding side. More specific techniques for the TS residual coding mode are described in Non-Patent Document 4, etc. The TS residual encoder 153 may encode the quantized coefficients in this manner and generate encoded data of the quantized coefficients. The TS residual encoder 153 may supply the generated encoded data to the selection unit 155.

[0103] The non-TS residual coding unit 154 performs processing related to the non-TS residual coding mode. For example, the non-TS residual coding unit 154 may acquire quantized coefficients supplied from the selection unit 152. The non-TS residual coding unit 154 may also encode the acquired quantized coefficients in the non-TS residual coding mode. This non-TS residual coding mode is a coding mode for when a transform process is performed. For example, the coding in the non-TS residual coding mode is optimized for coefficient data that has undergone a transform process, such as by scanning from the last coefficient of the coefficient data toward the DC component and transmitting position information of the last coefficient to the decoding side. More specific techniques for the non-TS residual coding mode are described in Non-Patent Document 4, etc. The non-TS residual coding unit 154 may encode the quantized coefficients in this manner and generate encoded data of the quantized coefficients. The non-TS residual coding unit 154 may supply the generated encoded data to the selection unit 155.

[0104] The selection unit 155 performs processing related to selection of a supply source of encoded data. For example, the selection unit 155 may acquire encoded data supplied from the TS residual encoding unit 153 or the non-TS residual encoding unit 154 (whichever is specified by the control unit 151). For example, when a transform skip is applied, the selection unit 155 acquires encoded data supplied from the TS residual encoding unit 153. Furthermore, for example, when a transform skip is not applied (when a transform process is performed), the selection unit 155 acquires encoded data supplied from the non-TS residual encoding unit 154. The selection unit 155 may supply the encoded data acquired in this manner to the accumulation buffer 116 (i.e., provide the data to the decoding side).

[0105] <Skip conversion of each component> The image coding device 100 can code a color image having a luminance component and a chrominance component. The image coding device 100 can code not only the luminance component but also the chrominance component by skipping (omitting) the conversion process that converts the residual between an image and its predicted image into coefficient data.

[0106] For example, the control unit 101 can set a transform skip flag (transform_skip_flag) indicating whether or not to apply a transform skip for each component. A component can be indicated by a component identifier (cIdx). That is, the control unit 101 can set the value of the transform skip flag in correspondence with the value of the component identifier.

[0107] That is, the control unit 101 can generate a transform skip flag (transform_skip_flag[xTbY][yTbY][cIdx]) indicating, for each component (cIdx), whether or not to skip the transform process for converting the residual between the image to be coded and its predicted image into coefficient data. Furthermore, the transform mode information coding unit 150 of the coding unit 115 can code the transform skip flag generated by the control unit 101 and generate coded data of the transform skip flag. Furthermore, the accumulation buffer 116 can generate a bitstream including the coded data of the transform skip flag generated by the transform mode information coding unit 150 and output the bitstream to the outside of the image coding device 100.

[0108] In this way, the control unit 101 can set a transform skip flag for, for example, a color component, and can also apply transform skip to the color component. Furthermore, the image encoding device 100 can provide the generated transform skip flag (i.e., the transform skip setting for each component) to the decoding side. Therefore, when the transform skip is applied to the color component, the decoding side can correctly decode the bitstream. Therefore, it is possible to suppress a decrease in encoding efficiency compared to when the transform skip is applicable only to the luminance component.

[0109] Furthermore, in the image encoding device 100, the transform processing of the color components may be controlled based on the setting of the transform skip for each component by the transform skip flag described above. For example, the orthogonal transform unit 113 may control the transform skip for each component based on the transform skip flag generated by the control unit 101. In this way, the orthogonal transform unit 113 can apply the transform skip to the transform processing of the color components, for example.

[0110] Note that, by setting the transform skip for each component as described above, the control unit 101 can set the transform skip for the color component independently of the luminance component. Therefore, the control unit 101 can set the transform skip for the color component depending on whether the transform skip is enabled for the color component. This allows the orthogonal transform unit 113 to perform the transform skip for the color component independently of the luminance component. In this way, it is possible to suppress a decrease in coding efficiency compared to when the transform skip setting for the luminance component is applied to the color component.

[0111] <Encoding mode control> Furthermore, the encoding unit 115 may control the encoding mode of the coefficient data corresponding to a component identifier based on a transform skip flag corresponding to the component identifier. For example, the control unit 151, the selection unit 152, and the selection unit 155 of the encoding unit 115 may control whether to use the TS residual encoding mode or the non-TS residual encoding mode based on the transform skip flag. For example, the TS residual encoding unit 153 and the non-TS residual encoding unit 154 may encode the coefficient data corresponding to the component identifier in the encoding mode set in this way, and generate encoded data of the coefficient data.

[0112] In other words, a plurality of candidate coding modes with different characteristics are prepared, and the coding unit 115 selects and applies a coding mode from the plurality of candidates based on a transform skip flag corresponding to a component identifier. That is, the coding unit 115 codes coefficient data in the selected coding mode. In this way, the coding unit 115 can apply a coding mode with characteristics more suitable for the setting of the transform skip among the coding modes with different characteristics, thereby suppressing a decrease in coding efficiency compared to when coding is performed using a single coding mode.

[0113] <Image encoding process flow> Next, a description will be given of the flow of each process executed by the above-described image encoding device 100. First, an example of the flow of image encoding processing will be described with reference to the flowchart in FIG.

[0114] When the image encoding process starts, in step S101, the reordering buffer 111 is controlled by the control unit 101 to reorder the frames of the input video data from the display order to the encoding order.

[0115] In step S102, the control unit 101 sets a processing unit for the input image held in the sorting buffer 111 (performs block division).

[0116] In step S103, the control unit 101 determines (sets) coding parameters for the input image held by the reordering buffer 111.

[0117] In step S104, the control unit 101 generates transform mode information (transform_mode) of the transform block corresponding to the component identifier (cIdx).

[0118] In step S105, the prediction unit 122 performs a prediction process to generate a predicted image etc. of an optimal prediction mode. For example, in this prediction process, the prediction unit 122 performs intra prediction to generate a predicted image etc. of an optimal intra prediction mode, performs inter prediction to generate a predicted image etc. of an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value etc.

[0119] In step S106, the calculation unit 112 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S105. That is, the calculation unit 112 generates a prediction residual D between the input image and the predicted image. The prediction residual D calculated in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to when the image is encoded as is.

[0120] In step S107, the orthogonal transform unit 113 performs an orthogonal transform process on the prediction residual D generated by the process of step S106, in accordance with the transform mode information generated in step S104, and derives the transform coefficient Coeff.

[0121] In step S108, the quantization unit 114 quantizes the transform coefficient Coeff obtained by the processing in step S107, for example, by using the quantization parameter calculated by the control unit 101, and derives the quantized transform coefficient level level.

[0122] In step S109, the inverse quantization unit 117 inverse quantizes the quantized transform coefficient level generated by the process of step S108 with characteristics corresponding to the quantization characteristics of step S108, to derive the transform coefficient Coeff_IQ.

[0123] In step S110, the inverse orthogonal transform unit 118 performs inverse orthogonal transform on the transform coefficients Coeff_IQ obtained by the process of step S109 using a method corresponding to the orthogonal transform process of step S107, in accordance with the transform mode information generated in step S104, to derive prediction residuals D'. Note that this inverse orthogonal transform process is similar to the inverse orthogonal transform process (described later) performed on the decoding side, and therefore the explanation (described later) about the decoding side can be applied to the inverse orthogonal transform process of step S110.

[0124] In step S111, the calculation unit 119 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S105 to the prediction residual D' derived in the process in step S110.

[0125] In step S112, the in-loop filter unit 120 performs in-loop filtering on the locally decoded image derived in the processing of step S111.

[0126] In step S113, the frame memory 121 stores the locally decoded image derived by the process in step S111 and the locally decoded image filtered in step S112.

[0127] In step S114, the encoding unit 115 encodes the quantized transform coefficient level LEVEL obtained by the process of step S108 and the transform mode information generated in step S104. For example, the encoding unit 115 encodes the quantized transform coefficient level LEVEL, which is information related to the image, by arithmetic coding or the like to generate encoded data. At this time, the encoding unit 115 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo). Furthermore, the encoding unit 115 derives residual information RInfo from the quantized transform coefficient level LEVEL and encodes the residual information RInfo.

[0128] In step S115, the accumulation buffer 116 accumulates the encoded data obtained in this manner and outputs it, for example, as a bit stream, to the outside of the image encoding device 100. This bit stream is transmitted to the decoding side, for example, via a transmission path or a recording medium.

[0129] In step S116, the rate control unit 123 performs rate control as necessary.

[0130] When the process of step S116 ends, the image encoding process ends.

[0131] <Encoding process flow> Next, an example of the flow of the encoding process executed in step S114 in FIG. 4 will be described with reference to the flowchart in FIG.

[0132] When the encoding process starts, in step S151, the transform mode information encoding unit 150 of the encoding unit 115 encodes the transform mode information (transform_mode) of the transform block corresponding to the component identifier cIdx.

[0133] In step S152, the control unit 151 derives Condition1 by the following formula (1). That is, the control unit 151 generates Condition1 using a transform skip flag (transform_skip_flag[cIdx]) corresponding to the component. Note that the transform skip flag (transform_skip_flag[cIdx]) may also include the coordinates (xTbY, yTbY) of the block to be processed (for example, transnform_skip_flag[xTbY][yTbY][cIdx]), but this is omitted here for simplicity. This will also be omitted hereinafter as appropriate.

[0134] TIFF2026012356000002.tif18132

[0135] In step S153, the control unit 151 determines whether Condition 1 is true. If Condition 1 is true, the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is true (IS_SKIP). Therefore, the selection unit 152 and the selection unit 155 are connected to the TS residual encoding unit 153 under the control of the control unit 151. As a result, the process proceeds to step S154.

[0136] In step S154, the TS residual encoding unit 153 encodes the quantized coefficients in the TS residual encoding mode to generate encoded data. Once the encoded data is generated, the encoding process ends.

[0137] Also, in step S153, if Condition1 is false, the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is false. Therefore, the selection unit 152 and the selection unit 155 connect to the non-TS residual encoding unit 154 under the control of the control unit 151. As a result, the process proceeds to step S155.

[0138] In step S155, the non-TS residual encoding unit 154 encodes the quantized coefficients in the non-TS residual encoding mode to generate encoded data. Once the encoded data is generated, the encoding process ends.

[0139] By performing the processes as described above, the image coding device 100 can apply the TS residual coding mode to color components as well, and can suppress a decrease in coding efficiency.

[0140] In the above description, whether or not a transform skip is performed is described as being notified by the transform skip flag (transform_skip_flag), but this is not limited thereto, and the information may be notified as one mode of the identifier mts_idx related to the primary transform. The identifier mts_idx is an identifier indicating the horizontal and vertical transform types of the primary transform. In this case, the control unit 151 can derive Condition1 as shown in the following equation (2).

[0141] TIFF2026012356000003.tif21118

[0142] <3-2. Image Decoding Device> Fig. 6 is a block diagram showing an example of the configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 200 shown in Fig. 6 is a device that decodes coded data in which a prediction residual between an image and its predicted image is coded, such as in AVC or HEVC. For example, the image decoding device 200 may implement the technology described in the above-mentioned non-patent documents and decode coded data in which image data of a moving image is coded using a method that complies with the standard described in one of those documents. For example, the image decoding device 200 may decode coded data (bitstream) generated by the above-mentioned image coding device 100.

[0143] Note that Fig. 6 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 6. In other words, in the image decoding device 200, there may be processing units that are not shown as blocks in Fig. 6, or there may be processing or data flows that are not shown as arrows, etc. in Fig. 6. This also applies to other figures that explain processing units, etc. in the image decoding device 200.

[0144] 6, the image decoding device 200 includes an accumulation buffer 211, a decoding unit 212, an inverse quantization unit 213, an inverse orthogonal transformation unit 214, a calculation unit 215, an in-loop filter unit 216, a rearrangement buffer 217, a frame memory 218, and a prediction unit 219. The prediction unit 219 includes an intra prediction unit and an inter prediction unit, which are not shown.

[0145] <Accumulation buffer> The accumulation buffer 211 acquires and holds (stores) the bitstream input to the image decoding device 200. At a predetermined timing, or when a predetermined condition is met, the accumulation buffer 211 extracts coded data included in the accumulated bitstream and supplies the coded data to the decoding unit 212.

[0146] <Decryption section> The decoding unit 212 performs processing related to image decoding. For example, the decoding unit 212 receives the coded data supplied from the accumulation buffer 211 as input, and performs entropy decoding (lossless decoding) on ​​the syntax values ​​of the syntax elements from the bit string in accordance with the definition of the syntax table, thereby deriving parameters.

[0147] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, etc. That is, the decoding unit 212 parses (analyzes and obtains) this information from the bitstream. These pieces of information will be described below.

[0148] <Header information Hinfo> The header information Hinfo includes header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH). The header information Hinfo includes information that specifies, for example, an image size (horizontal width PicWidth, vertical width PicHeight), a bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum CU size (MaxCUSize) and a minimum CU size (MinCUSize), a maximum depth MaxQTDepth and a minimum depth MinQTDepth of quad-tree partitioning (also referred to as quad-tree partitioning), a maximum depth MaxBTDepth and a minimum depth MinBTDepth of binary-tree partitioning, a maximum transform skip block size MaxTSSize (also referred to as maximum transform skip block size), an on / off flag (also referred to as a valid flag) of each encoding tool, and the like.

[0149] For example, the on / off flags for the encoding tools included in the header information Hinfo include on / off flags related to the following conversion and quantization processes. Note that the on / off flags for the encoding tools can also be interpreted as flags indicating whether or not syntax related to the encoding tool is present in the encoded data. Furthermore, if the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable, and if the value of the on / off flag is 0 (false), it indicates that the encoding tool is unusable. Note that the interpretation of the flag values ​​may be reversed.

[0150] Inter-component prediction enabled flag (ccp_enabled_flag): Flag information indicating whether inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available. For example, if this flag information is "1" (true), it indicates that it is available, and if it is "0" (false), it indicates that it is not available.

[0151] This CCP is also called cross-component linear prediction (CCLM or CCLMP).

[0152] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and the like.

[0153] The intra prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from the syntax.

[0154] In addition, the intra-prediction mode information IPinfo includes, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chrominance sample position type identifier (chroma_sample_loc_type_idx), a chrominance MPM identifier (chroma_mpm_idx), and a luminance intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0155] The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether or not inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0156] The multi-class linear prediction mode flag (mclm_flag) is information related to the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single-class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0157] The chrominance sample location type identifier (chroma_sample_loc_type_idx) is an identifier that identifies the type of pixel location of the chrominance component (also referred to as the chrominance sample location type).

[0158] Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted (stored in) information about the pixel location of the chrominance component (chroma_sample_loc_info()).

[0159] The chrominance MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chrominance intra prediction mode.

[0160] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0161] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0162] <Conversion information Tinfo> The conversion information Tinfo includes, for example, the following information: Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0163] The width size TBWSize and height TBHSize of the transform block to be processed (or the logarithmic values ​​log2TBWSize and log2TBHSize of TBWSize and TBHSize in base 2). Transform skip flag (ts_flag): A flag indicating whether to skip the (inverse) primary transform and (inverse) secondary transform. Scan Identifier (scanIdx) Quantization parameter (qp) Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax))

[0164] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) includes, for example, the following syntax:

[0165] cbf(coded_block_flag): residual data presence flag last_sig_coeff_x_pos: Last non-zero coefficient X coordinate last_sig_coeff_y_pos: Last non-zero coefficient Y coordinate coded_sub_block_flag: Sub-block non-zero coefficient presence flag sig_coeff_flag: Non-zero coefficient presence flag gr1_flag: Flag indicating whether the level of the non-zero coefficient is greater than 1 (also called the GR1 flag) gr2_flag: Flag indicating whether the level of the non-zero coefficient is greater than 2 (also called GR2 flag) sign_flag: The sign indicating the positive or negative value of the non-zero coefficient (also called the sign sign) coeff_abs_level_remaining: Non-zero coefficient residual level (also called non-zero coefficient residual level) etc.

[0166] Of course, the residual information Rinfo may include any information, and may include information other than the above information.

[0167] <Filter information Finfo> The filter information Finfo includes, for example, control information related to each of the following filter processes:

[0168] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0169] More specifically, for example, the information includes information specifying the picture to which each filter is applied, an area within the picture, filter On / Off control information for each CU, filter On / Off control information related to slice and tile boundaries, etc. Of course, any information may be included in the filter information Finfo, and information other than these may be included.

[0170] Returning to the description of the decoding unit 212, the decoding unit 212 derives the quantized transform coefficient level "level" at each coefficient position in each transform block by referring to the residual information Rinfo. The decoding unit 212 supplies the quantized transform coefficient level to the inverse quantization unit 213.

[0171] Furthermore, the decoding unit 212 supplies the parsed header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, and filter information Finfo to each block.

[0172] The header information Hinfo is supplied to the inverse quantization unit 213, the inverse orthogonal transform unit 214, the prediction unit 219, and the in-loop filter unit 216. The prediction mode information Pinfo is supplied to the inverse quantization unit 213 and the prediction unit 219. The transformation information Tinfo is supplied to the inverse quantization unit 213 and the inverse orthogonal transform unit 214. The filter information Finfo is supplied to the in-loop filter unit 216.

[0173] Of course, the above example is merely an example and is not limiting. For example, each encoding parameter may be supplied to any processing unit. Furthermore, other information may be supplied to any processing unit.

[0174] <Inverse quantization section> The inverse quantization unit 213 performs processing related to inverse quantization. For example, the inverse quantization unit 213 receives the transformation information Tinfo and the quantized transformation coefficient level level supplied from the decoding unit 212 as input, scales (inverse quantizes) the value of the quantized transformation coefficient level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization.

[0175] Note that this inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 114. Furthermore, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 117. That is, the inverse quantization unit 117 performs the same process (inverse quantization) as the inverse quantization unit 213.

[0176] The inverse quantization unit 213 supplies the derived transform coefficient Coeff_IQ to the inverse orthogonal transformation unit 214 .

[0177] <Inverse orthogonal transform section> The inverse orthogonal transform unit 214 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 214 receives the transform coefficients Coeff_IQ supplied from the inverse quantization unit 213 and the transform information Tinfo supplied from the decoding unit 212 as input, and performs inverse orthogonal transform processing (inverse transform processing) on ​​the transform coefficients based on the transform information Tinfo to derive a prediction residual D'.

[0178] This inverse orthogonal transform is performed as the inverse process of the orthogonal transform performed by the orthogonal transform unit 113. This inverse orthogonal transform is the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 118. That is, the inverse orthogonal transform unit 118 performs the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 214.

[0179] The inverse orthogonal transform unit 214 supplies the derived prediction residual D′ to the calculation unit 215.

[0180] <Arithmetic section> The calculation unit 215 performs processing related to addition of information related to an image. For example, the calculation unit 215 receives as input a prediction residual supplied from the inverse orthogonal transform unit 214 and a predicted image supplied from the prediction unit 219. The calculation unit 215 adds the prediction residual and a predicted image (prediction signal) corresponding to the prediction residual, and derives a locally decoded image.

[0181] The calculation unit 215 supplies the derived locally decoded image to the in-loop filter unit 216 and the frame memory 218 .

[0182] <In-loop filter section> The in-loop filter unit 216 performs processing related to in-loop filter processing. For example, the in-loop filter unit 216 receives as input the locally decoded image supplied from the calculation unit 215 and filter information Finfo supplied from the decoding unit 212. Note that any information may be input to the in-loop filter unit 216, and information other than the above information may also be input.

[0183] The in-loop filter unit 216 performs appropriate filtering on the locally decoded image based on the filter information Finfo.

[0184] For example, the in-loop filter unit 216 applies four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate.

[0185] The in-loop filter unit 216 performs a filter process corresponding to the filter process performed on the encoding side (for example, the in-loop filter unit 120 of the image encoding device 100). Of course, the filter process performed by the in-loop filter unit 216 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 216 may apply a Wiener filter or the like.

[0186] The in-loop filter unit 216 supplies the filtered locally decoded image to a reordering buffer 217 and a frame memory 218 .

[0187] <Sorting buffer> The reordering buffer 217 receives the locally decoded images supplied from the in-loop filter unit 216 as input and holds (stores) them. The reordering buffer 217 uses the locally decoded images to reconstruct decoded images for each picture and holds them (stores them in the buffer). The reordering buffer 217 reorders the obtained decoded images from decoding order to playback order. The reordering buffer 217 outputs the reordered decoded images as video data to the outside of the image decoding device 200.

[0188] <Frame memory> The frame memory 218 performs processing related to the storage of image data. For example, the frame memory 218 receives the locally decoded image supplied from the calculation unit 215 as input, reconstructs the decoded image for each picture, and stores the reconstructed image in a buffer within the frame memory 218.

[0189] The frame memory 218 also receives as input the locally decoded image that has been subjected to in-loop filtering and is supplied from the in-loop filter unit 216, reconstructs a decoded image for each picture, and stores the reconstructed image in a buffer within the frame memory 218. The frame memory 218 appropriately supplies the stored decoded image (or a part thereof) to the prediction unit 219 as a reference image.

[0190] The frame memory 218 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to the generation of decoded images.

[0191] <Encoding part> Fig. 7 is a block diagram showing an example of the main configuration of the decoding unit 212 in Fig. 6. As shown in Fig. 7, the decoding unit 212 has a transform mode information decoding unit 250, a control unit 251, a selection unit 252, a TS residual decoding unit 253, a non-TS residual decoding unit 254, and a selection unit 255.

[0192] The transform mode information decoding unit 250 performs processing related to decoding of encoded data of transform mode information (transform_mode). This transform mode information is information related to the mode of inverse transform processing by the inverse orthogonal transform unit 214. For example, the transform mode information may include a transform skip flag (transform_skip_flag[xTbY][yTbY][cIdx]), an identifier related to primary transform (mts_idx[xTbY][yTbY][cIdx]), etc.

[0193] For example, the transform mode information decoding unit 250 may acquire coded data supplied from the accumulation buffer 211. The transform mode information decoding unit 250 may also decode the acquired coded data to generate transform mode information and a component identifier (cIdx). Furthermore, the transform mode information encoding unit 150 may supply the generated transform mode information, etc. to the control unit 251.

[0194] The control unit 251 performs processing related to decoding mode control. For example, the control unit 251 may acquire transform mode information (e.g., transform_skip_flag[xTbY][yTbY][cIdx]) or mts_idx[xTbY][yTbY][cIdx]) or a component identifier (cIdx) supplied from the transform mode information decoding unit 250. Furthermore, the control unit 251 may switch the decoding mode of the coded data of the coefficient data between a TS residual decoding mode and a non-TS residual decoding mode by controlling the selection of the selection unit 252 or the selection unit 255 based on the transform skip flag corresponding to the component identifier. For example, when a transform skip is applied, the control unit 251 connects the selection unit 252 and the selection unit 255 to the TS residual decoding unit 253. Furthermore, for example, when a transform skip is not applied (when a transform process is performed), the control unit 251 connects the selection unit 252 and the selection unit 255 to the non-TS residual decoding unit 254.

[0195] The selection unit 252 performs processing related to selecting a supply destination of coded data of coefficient data (quantized coefficients). For example, the selection unit 252 may acquire coded data supplied from the accumulation buffer 211. Furthermore, under the control of the control unit 251, the selection unit 252 may supply the acquired coded data to the TS residual decoding unit 253 or the non-TS residual decoding unit 254 (the one designated by the control unit 251). For example, when transform skip is applied, the selection unit 252 supplies the coded data to the TS residual decoding unit 253. Furthermore, for example, when transform skip is not applied (when transform processing has been performed), the selection unit 252 supplies the coded data to the non-TS residual decoding unit 254.

[0196] The TS residual decoding unit 253 performs processing related to the TS residual decoding mode. For example, the TS residual decoding unit 253 may acquire coded data supplied from the selection unit 252. The TS residual decoding unit 253 may also decode the acquired coded data in the TS residual decoding mode. This TS residual decoding mode is a decoding mode for skipping the transform process. For example, decoding in the TS residual decoding mode corresponds to coding in the TS residual coding mode and is optimized for coded data of coefficient data for which the transform process has been skipped. More specific techniques for the TS residual decoding mode are described in Non-Patent Document 4, etc. The TS residual decoding unit 253 may decode the coded data in this manner and generate quantized coefficients. The TS residual decoding unit 253 may supply the generated quantized coefficients to the selection unit 255.

[0197] The non-TS residual decoding unit 254 performs processing related to the non-TS residual decoding mode. For example, the non-TS residual decoding unit 254 may acquire coded data supplied from the selection unit 252. The non-TS residual decoding unit 254 may also decode the acquired coded data in the non-TS residual decoding mode. This non-TS residual decoding mode is a decoding mode for when a transform process is performed. For example, decoding in the non-TS residual decoding mode corresponds to coding in the non-TS residual coding mode and is optimized for coded data of coefficient data that has undergone the transform process. More specific techniques for the non-TS residual decoding mode are described in Non-Patent Document 4, etc. The non-TS residual decoding unit 254 may decode coded data in this manner and generate quantized coefficients. The non-TS residual decoding unit 254 may supply the quantized coefficients generated in this manner to the selection unit 255.

[0198] The selection unit 255 performs processing related to selection of a supply source of quantization coefficients. For example, the selection unit 255 may acquire quantization coefficients supplied from the TS residual decoding unit 253 or the non-TS residual decoding unit 254 (whichever is specified by the control unit 251). For example, when a transform skip is applied, the selection unit 255 acquires quantization coefficients supplied from the TS residual decoding unit 253. Furthermore, for example, when a transform skip is not applied (when a transform process is performed), the selection unit 255 acquires coded data supplied from the non-TS residual decoding unit 254. The selection unit 255 may supply the quantization coefficients acquired in this manner to the inverse quantization unit 213.

[0199] <Skip conversion of each component> The image decoding device 200 can decode coded data of a color image having a luminance component and a chrominance component. The image decoding device 200 can also skip (omit) the inverse transform process for converting coefficient data generated by decoding coded data of not only the luminance component but also the chrominance component into residuals between an image and its predicted image.

[0200] For example, the transform mode information decoding unit 250 can decode the encoded data of the transform skip flag corresponding to the component identifier, and obtain the transform skip flag corresponding to that component identifier.

[0201] By doing so, it is possible to apply the transform skip setting for each component indicated by the transform skip flag during decoding. Therefore, it is possible to correctly decode the bitstream of the color component to which the transform skip is applied. Therefore, it is possible to suppress a decrease in coding efficiency compared to when the transform skip is applicable only to the luminance component.

[0202] Furthermore, when decoding coded image data, the inverse transform process of a color component may be controlled based on the setting of the transform skip for each component using the transform skip flag described above. That is, the transform skip may be applied to the inverse transform process of a color component. By doing so, it is possible to correctly decode the bit stream of the color component to which the transform skip has been applied. Therefore, it is possible to suppress a decrease in coding efficiency compared to when the transform skip is applicable only to the luminance component.

[0203] As described above, by setting the transform skip for each component, it is possible to set the transform skip for the color component independently of the luminance component. Therefore, it is possible to set the transform skip for the color component depending on whether the transform skip is enabled for the color component. In this way, it is possible to suppress a decrease in coding efficiency compared to when the transform skip setting for the luminance component is applied to the color component.

[0204] <Encoding mode control> Furthermore, such control of the decoding mode may also be applied when applying transform skip to color components as in the above-described method 1. For example, when applying transform skip to each value of the component identifier, the TS residual decoding mode may be applied as the decoding mode, and when the transform skip is not applied, that is, when non-transform skip is applied, the non-TS residual decoding mode may be applied as the decoding mode.

[0205] For example, based on a transform skip flag corresponding to a component identifier, the decoding mode of the coded data of the coefficient data corresponding to that component identifier may be controlled to either a TS residual decoding mode, which is a mode for skipping the inverse transform process of converting the coefficient data into the residual between an image and a predicted image, or a non-TS residual decoding mode, which is a mode for not skipping the inverse transform process, and the coded data of the coefficient data corresponding to the component identifier may be decoded using the decoding mode set in this way to generate coefficient data corresponding to that component identifier.

[0206] By doing so, it is possible to apply a decoding mode according to the setting of the transform skip, and therefore it is possible to suppress a decrease in coding efficiency compared to when decoding is performed using a single decoding mode.

[0207] <Flow of image decoding process> Next, a description will be given of the flow of each process executed by the image decoding device 200 configured as above. First, an example of the flow of image decoding process will be described with reference to the flowchart in FIG.

[0208] When the image decoding process starts, in step S401, the accumulation buffer 211 acquires and holds (accumulates) a bitstream supplied from outside the image decoding device 200.

[0209] In step S202, the decoding unit 212 extracts and decodes the coded data from the bitstream to obtain the quantized transform coefficient level. Furthermore, the decoding unit 212 parses (analyzes and obtains) various coding parameters from the bitstream through this decoding.

[0210] In step S203, the inverse quantization unit 213 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S202 to obtain a transform coefficient Coeff_IQ.

[0211] In step S204, the inverse orthogonal transform unit 214, under the control of step S203, performs inverse orthogonal transform processing, which is the inverse processing of the orthogonal transform processing performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S203, to obtain a prediction residual D'.

[0212] In step S205, the prediction unit 219 performs prediction processing using a prediction method specified by the encoding side based on the information parsed in step S202, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 218.

[0213] In step S206, the calculation unit 215 adds the prediction residual D′ obtained in step S204 and the predicted image P obtained in step S205 to obtain a locally decoded image R local is derived.

[0214] In step S207, the in-loop filter unit 216 filters the locally decoded image R obtained by the processing in step S206. local In-loop filtering is performed on

[0215] In step S208, the sorting buffer 217 sorts the filtered locally decoded image R obtained by the processing in step S207. local The decoded images R are derived using the above formula, and the order of the decoded images R is rearranged from the decoding order to the playback order. The decoded images R rearranged in the playback order are output to the outside of the image decoding device 200 as moving images.

[0216] In step S209, the frame memory 218 stores the locally decoded image R obtained by the process in step S206. local , and the filtered locally decoded image R obtained by the processing in step S207. local At least one of the above is stored.

[0217] When the process of step S209 is completed, the image decoding process ends.

[0218] <Decryption process flow> Next, an example of the flow of the decoding process executed in step S202 in FIG. 8 will be described with reference to the flowchart in FIG.

[0219] When the decoding process starts, in step S251, the transform mode information decoding unit 250 of the decoding unit 212 decodes the transform mode information (transform_mode) of the transform block corresponding to the component identifier cIdx.

[0220] In step S252, the control unit 251 derives Condition 1 using the above-mentioned formula (1). That is, the control unit 251 generates Condition 1 using the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component.

[0221] In step S253, the control unit 251 determines whether or not Condition 1 is true. If Condition 1 is true, the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is true (IS_SKIP). Therefore, the selection unit 252 and the selection unit 255 are connected to the TS residual decoding unit 253 under the control of the control unit 251. As a result, the process proceeds to step S254.

[0222] In step S254, the TS residual decoding unit 253 decodes the coded data in the TS residual decoding mode to generate quantized coefficients. Once the quantized coefficients are generated, the decoding process ends.

[0223] Also, in step S253, if Condition1 is false, the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is false. Therefore, the selection unit 252 and the selection unit 255 connect to the non-TS residual decoding unit 254 under the control of the control unit 251. As a result, the process proceeds to step S255.

[0224] In step S255, the non-TS residual decoding unit 254 encodes the encoded data in non-TS residual decoding mode to generate quantized coefficients. Once the quantized coefficients are generated, the decoding process ends.

[0225] By performing the processes as described above, the image decoding device 200 can apply the TS residual decoding mode to color components as well, and can suppress a decrease in coding efficiency.

[0226] In the above description, whether or not a transform skip is performed is described as being notified by the transform skip flag (transform_skip_flag), but this is not limiting and the information may be notified as one mode of the identifier mts_idx related to the primary transform. The identifier mts_idx is an identifier indicating the horizontal and vertical transform types of the primary transform. In this case, the control unit 251 can derive Condition1 as shown in the above-described formula (2).

[0227] <3-3. Syntax and Semantics> An example of the syntax of a TU (transform unit) in this case is shown in Figure 10. In the example of Figure 10, the transform mode information (transform_mode(... ,0), transform_mode(... ,0), transform_mode(... ,2)) corresponding to each component is derived, and whether to apply a transform skip (residual_coding(... ,cIdx) or residual_ts_coding(... ,cIdx)) is determined for each component (tu_cbf_luma, tu_cbf_cb, tu_cbf_cr).

[0228] An example of the syntax of the transform mode information is shown in FIG. 11. In the example of FIG. 11, the transform skip flag for each component (transform_skip_flag[x0][y0][cIdx]) and the transform type identifier of the primary transform for each component (mts_idx[x0][y0][cIdx]) are set as the transform mode information for each component (transform_mode(...,cIdx)). In this case, the conditions for generating this information for each component are the same. The content of this condition is arbitrary. For example, the transform skip enable flag (sps_transform_skip_enabled_flag) set in the sequence parameter set (SPS) can be used as the condition.

[0229] An example of the syntax of transform skip (residual_ts_coding(... ,cIdx)) is shown in Figure 12. As shown in Figure 12, the transform skip is set for each component (cIdx). Also, an example of the semantics of the transform skip flag (transform_skip_flag[x0][y0][cIdx]) of the transform mode information is shown in A of Figure 13. Furthermore, an example of the semantics of the transform type identifier (mts_idx[x0][y0][cIdx]) of the primary transform is shown in B of Figure 13. In this way, information about the transform skip is set for each component. Therefore, as described above, the transform skip can be applied to color components, and a decrease in coding efficiency can be suppressed.

[0230] In the syntax of the transform mode information, the conditions for each component to generate the transform skip flag (transform_skip_flag[x0][y0][cIdx]) for each component and the transform type identifier (mts_idx[x0][y0][cIdx]) for the primary transform for each component may be different from each other, as in the example of Fig. 14. By providing conditions for each component in this way, it is possible to reduce the redundancy of the judgment conditions and prevent an increase in the processing load.

[0231] <3-4. Transform skip residual coding use flag> Note that a transform skip residual coding use flag may be applied to indicate selection between the TS residual coding mode (TS residual decoding mode) and the non-TS residual coding mode (non-TS residual decoding mode).

[0232] An example of the syntax of the sequence parameter set (SPS) is shown in Fig. 15. For example, the control unit 101 may set a transform skip residual coding use flag sps_ts_residual_coding_use_flag in this sequence parameter set.

[0233] This sps_ts_residual_coding_use_flag is a transform skip residual coding use flag notified at the sequence parameter set level. If the value of this flag is "1", it indicates that the TS residual coding mode is applied when a transform skip occurs. If the value of this flag is "0", it indicates that the non-TS residual coding mode is applied when a transform skip occurs.

[0234] An example of the syntax of a TU in this case is shown in Figure 16. In the example of Figure 16, this transform skip residual coding use flag is used as a condition for determining whether to apply transform skip. The semantics of sps_ts_residual_coding_use_flag are shown in Figure 17. In this case, Condition1 is derived using, for example, the following equation (3).

[0235] TIFF2026012356000004.tif22131

[0236] When using the conversion type identifier of the primary conversion, it is derived as shown in (4) below.

[0237] TIFF2026012356000005.tif24133

[0238] By using such a high-level flag, even encoders and decoders that do not support transform skip can correctly perform encoding and decoding based on this flag. Therefore, it is possible to suppress a decrease in encoding efficiency. In other words, it is possible to omit the implementation of the TS residual encoding mode (TS residual decoding mode) in the encoder and decoder, thereby suppressing an increase in circuit scale.

[0239] The data unit for setting the transform skip residual coding use flag may be any data unit other than the sequence parameter set. For example, the transform skip residual coding use flag may be notified at the CU level, slice level, picture level, etc. The finer the data unit granularity, the greater the degree of freedom in mode switching and the greater the room for improving coding efficiency.

[0240] <3-5. Transform skip remaining encoding use specific mode flag> Furthermore, a flag may be applied to indicate whether a TS residual coding mode (TS residual decoding mode) or a non-TS residual coding mode (non-TS residual decoding mode) is to be applied in a particular mode.

[0241] An example of the syntax of the sequence parameter set (SPS) is shown in Fig. 18. For example, the control unit 101 may set a transform skip residual coding use specific mode flag sps_ts_residual_coding_use_for_bdpcm_flag in this sequence parameter set.

[0242] This sps_ts_residual_coding_use_for_bdpcm_flag is a flag notified at the sequence parameter set level, and is flag information indicating that coding mode selection is enabled for BDPCM (Block-based Differential Pulse Code Modulation). When this flag value is "1", it indicates that the TS residual coding mode is applied for BDPCM. When this flag value is "0", it indicates that the non-TS residual coding mode is applied for BDPCM.

[0243] An example of the syntax of the TU in this case is shown in Figure 19. In the example of Figure 19, this transform skip residual coding use specific mode flag is used as a condition for determining whether to apply transform skip. The semantics of sps_ts_residual_coding_use_for_bdpcm_flag are shown in Figure 20. In this case, Condition1 is derived using, for example, the following equation (5).

[0244] TIFF2026012356000006.tif17141

[0245] When using the conversion type identifier of the primary conversion, it is derived as shown in (6) below.

[0246] TIFF2026012356000007.tif20141

[0247] By using such a high-level flag, it becomes possible to switch between TS residual coding mode and non-TS residual coding mode in BDPCM.

[0248] The data unit for setting the transform skip / residual coding use specific mode flag may be any data unit other than the sequence parameter set. For example, the transform skip / residual coding use specific mode flag may be notified at the CU level, slice level, picture level, etc. The finer the data unit granularity, the greater the degree of freedom in mode switching and the greater the room for improving coding efficiency.

[0249] 4. Second Embodiment <4-1. Correction of quantization parameters in quantization> For each value of the component identifier, in the case of a transform skip, the quantization parameter may be corrected. That is, when a transform skip is applied, the quantization parameter may be corrected. Such control may be performed for each component.

[0250] <Quantization section> In this case, the image coding device 100 is the same as the example in Fig. 2. Fig. 21 is a block diagram showing an example of the main configuration of the quantization unit 114 in this case. As shown in Fig. 21, the quantization unit 114 in this case includes a QP correction unit 311 and a quantization processing unit 312.

[0251] The QP correction unit 311 performs processing related to correction of a quantization parameter. For example, the QP correction unit 311 can derive a quantization parameter Qp to be applied to a target transform block corresponding to the component identifier cIdx by referring to a transform skip flag corresponding to the component identifier cIdx, joint chroma encoding mode information TuResMode, a CU-level QP (Qp') corresponding to the component identifier cIdx, and a minimum QP (QpPrimeTsMin) of the transform skip. Types of this quantization parameter include Qp'y, Qp'cb, Qp'cr, and Qp'cbcr. Furthermore, the minimum quantization parameter QpPrimeTsMin of the transform skip is notified in a parameter set.

[0252] The joint chrominance encoding mode is a mode in which only one of the chrominance components (Cb, Cr) is transmitted, and the other is derived from the other and not transmitted. For example, only the Cr residual is encoded and transmitted, and the Cb residual is derived from the Cr residual and not transmitted. The joint chrominance encoding mode information TuResMode is information related to such a joint chrominance encoding mode.

[0253] The QP correction unit 311 supplies the corrected quantization parameter (corrected QP(qP)) to the quantization processing unit 312.

[0254] The quantization processing unit 312 quantizes the coefficient data (transform coefficients) using the quantization parameter (corrected QP(qP)) supplied from the QP correction unit 311, and generates quantized coefficients. The quantization processing unit 312 supplies the generated quantized coefficients to the encoding unit 115 and the inverse quantization unit 117.

[0255] <Quantization process flow> An example of the flow of the quantization process executed in step S108 of Fig. 4 in this case will be described with reference to the flowchart of Fig. 22. When the quantization process starts, the QP correction unit 311 of the quantization unit 114 derives Condition2 using the following equation (7) in step S301. That is, the QP correction unit 311 generates Condition2 using the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component.

[0256] TIFF2026012356000008.tif17130

[0257] When the conversion type identifier of the primary conversion is used, it is derived as shown in (8) below.

[0258] TIFF2026012356000009.tif20113

[0259] In step S302, the QP corrector 311 determines whether or not Condition 2 is true. If Condition 2 is true, that is, if the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is true (IS_SKIP), the process proceeds to step S303.

[0260] In step S303, the QP correction unit 311 corrects the quantization parameter QP. In this case, the QP correction unit 311 sets, for example, the larger of the minimum quantization parameter (QpPrimeTsMin) of the transform skip and the quantization parameter QP'x at the CU level as the quantization parameter (corrected QP(qP)) to be applied to the transform block to be processed corresponding to the component identifier. When the process of step S303 ends, the process proceeds to step S305.

[0261] On the other hand, if it is determined that Condition2 is false, the process proceeds to step S304. In this case, the QP correction unit 311 sets, for example, a quantization parameter QP'x at the CU level as the quantization parameter (corrected QP(qP)) to be applied to the transform block to be processed corresponding to the component identifier. When the process of step S304 ends, the process proceeds to step S305.

[0262] In step S305, the quantization processing unit 312 quantizes the transform coefficients using the quantization parameters updated in step S303 or step S304. When the process of step S305 ends, the quantization process ends and the process returns to FIG.

[0263] By doing so, it is possible to suppress a decrease in PSNR.

[0264] <4-2. Correction of quantization parameters in inverse quantization> Similarly, in inverse quantization, for each value of the component identifier, in the case of a transform skip, the quantization parameter may be corrected. That is, when a transform skip is applied, the quantization parameter may be corrected. Such control may be performed for each component.

[0265] <Inverse quantization section> In this case, the image decoding device 200 is the same as the example in Fig. 6. Fig. 23 is a block diagram showing an example of the main configuration of the inverse quantization unit 213 in this case. As shown in Fig. 23, the inverse quantization unit 213 in this case includes a QP correction unit 411 and an inverse quantization processing unit 412.

[0266] The QP correction unit 411 performs processing related to correction of quantization parameters. For example, the QP correction unit 411 can derive a quantization parameter Qp to be applied to a target transform block corresponding to the component identifier cIdx by referring to a transform skip flag corresponding to the component identifier cIdx, joint chroma encoding mode information TuResMode, a CU-level QP (Qp') corresponding to the component identifier cIdx, and a minimum QP (QpPrimeTsMin) of the transform skip. Types of this quantization parameter include Qp'y, Qp'cb, Qp'cr, and Qp'cbcr. Furthermore, the minimum quantization parameter QpPrimeTsMin of the transform skip is notified in a parameter set.

[0267] The QP correction unit 411 supplies the corrected quantization parameter (corrected QP(qP)) to the inverse quantization processing unit 412.

[0268] The inverse quantization unit 412 inverse-quantizes the quantized coefficients using the quantization parameter (corrected QP(qP)) supplied from the QP correction unit 411, and generates coefficient data (transform coefficients). The inverse quantization unit 412 supplies the generated coefficient data to the inverse orthogonal transform unit 214.

[0269] <Flow of inverse quantization process> An example of the flow of the inverse quantization process executed in step S203 of Fig. 8 in this case will be described with reference to the flowchart of Fig. 24. When the inverse quantization process starts, the QP correction unit 411 of the inverse quantization unit 213 derives Condition2 using the above-mentioned equation (7) in step S401. That is, the QP correction unit 411 generates Condition2 using the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component. This transform skip flag is obtained by decoding by the transform mode information decoding unit 250, as described in the first embodiment.

[0270] In step S402, the QP corrector 411 determines whether or not Condition 2 is true. If Condition 2 is true, that is, if the transform skip flag (transform_skip_flag[cIdx]) corresponding to the component is true (IS_SKIP), the process proceeds to step S403.

[0271] In step S403, the QP correction unit 411 corrects the quantization parameter QP. In this case, the QP correction unit 411 sets, for example, the larger of the minimum quantization parameter (QpPrimeTsMin) of the transform skip and the quantization parameter QP'x at the CU level as the quantization parameter (corrected QP(qP)) to be applied to the transform block to be processed corresponding to the component identifier. When the process of step S403 ends, the process proceeds to step S405.

[0272] On the other hand, if it is determined that Condition2 is false, the process proceeds to step S404. In this case, the QP correction unit 411 sets, for example, a quantization parameter QP'x at the CU level as the quantization parameter (corrected QP(qP)) to be applied to the transform block to be processed corresponding to the component identifier. When the process of step S404 ends, the process proceeds to step S405.

[0273] In step S405, the inverse quantization unit 412 inverse quantizes the quantized coefficients using the quantization parameters updated in step S403 or step S404. When the process of step S405 ends, the inverse quantization process ends and the process returns to FIG.

[0274] By doing so, it is possible to suppress a decrease in PSNR.

[0275] <4-3.Syntax> An example of the syntax of the quantization parameter in this case is shown in Fig. 25. As described above, the quantization parameter qP is corrected for each component using the transform skip flag transform_skip_flag.

[0276] 5. Third Embodiment <Sharing context variables> When the conversion skip is applied as described above, the context variables corresponding to each binIdx in the bin string of each syntax may be shared between the luminance component and the color component.

[0277] For example, when the transform process is skipped during encoding, the context variables may be shared between the encoding of the luminance component of the coefficient data and the encoding of the chrominance component.

[0278] Furthermore, for example, when inverse transform processing is skipped during decoding, the context variables may be shared between the decoding of coded data of the luminance component of coefficient data and the decoding of coded data of the chrominance component.

[0279] An example of syntax related to context variables in this case is shown in Fig. 26. When transform skip is applied as in the example of Fig. 26, context variables may be derived using a method common to the luminance component and the color components.

[0280] Another example of the syntax related to context variables is shown in Fig. 27. When a transform skip is applied as in the example of Fig. 27, the context variables for the luminance component and the color component may be derived using methods independent of each other.

[0281] As described above, by sharing the context variables corresponding to each binIdx in the bin string of each syntax between the luminance component and the color component, it is possible to suppress an increase in the memory size for storing the context variables, thereby suppressing an increase in hardware costs.

[0282] 6. Fourth Embodiment <Sign code encoding / decoding mode control> Also, the encoding / decoding method of the sign code may be switched depending on the transform skip flag corresponding to the component identifier. An example of the syntax in this case is shown in FIG.

[0283] As shown in the column of binIdx = 0 in the table of Fig. 28, for example, in the case of no transform skip, bypass coding / decoding may be applied to the coding / decoding of the sign code. Also, for example, in the case of transform skip, if the number of remaining context coding bins is equal to or greater than a threshold, context coding / decoding may be applied to the coding / decoding of the sign code, and otherwise bypass coding / decoding may be applied to the coding / decoding of the sign code ((MaxCcbs > 0) ? (0...5) : bypass).

[0284] By doing so, it is possible to suppress a decrease in coding efficiency.

[0285] <7. Notes> <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0286] FIG. 29 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0287] In a computer 900 shown in FIG. 29, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0288] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0289] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0290] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0291] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0292] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0293] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0294] <Unit of information and processing> The data units in which the various pieces of information described above are set and the data units targeted by the various processes are each arbitrary and are not limited to the above examples. For example, these pieces of information and processes may be set for each TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), sub-block, block, tile, slice, picture, sequence, or component, or may target data of these data units. Of course, these data units may be set for each piece of information or process, and the data units for all pieces of information and processes do not need to be unified. Note that the storage location of these pieces of information is arbitrary and may be stored in the headers or parameter sets of the above-mentioned data units, or may be stored in multiple locations.

[0295] <Control information> Control information related to the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side. For example, control information (e.g., enabled_flag) that controls whether or not to permit (or prohibit) application of the above-described present technology may be transmitted. Also, for example, control information (e.g., present_flag) that indicates a target to which the above-described present technology is to be applied (or a target to which it is not to be applied) may be transmitted. For example, control information that specifies a block size (upper or lower limit, or both), frame, component, or layer to which the present technology is to be applied (or permitted or prohibited to be applied) may be transmitted.

[0296] <Applicable targets of this technology> This technology can be applied to any image encoding / decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding / decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. Furthermore, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0297] This technology can also be applied to a multi-viewpoint image encoding / decoding system that encodes and decodes multi-viewpoint images that include images from multiple views. In this case, this technology can be applied to the encoding and decoding of each view.

[0298] Furthermore, this technology can be applied to a layered image coding (scalable coding) and decoding system that encodes and decodes layered images with multiple layers to provide scalability for predetermined parameters. In this case, this technology can be applied to the encoding and decoding of each layer.

[0299] Furthermore, although the image encoding device 100 and the image decoding device 200 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0300] For example, this technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) used in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communications, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0301] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0302] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0303] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0304] <Fields and applications where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0305] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0306] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0307] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0308] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0309] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0310] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0311] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0312] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0313] For example, the steps of a program executed by a computer may be executed in chronological order in the order described herein, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the steps may be executed in an order different from the order described above. Furthermore, the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0314] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0315] The present technology can also be configured as follows. (1) a flag generating unit that generates a transform skip flag, which is flag information indicating whether or not to skip a transform process for transforming a residual between an image and a predicted image of the image into coefficient data, for each component in encoding the image; a flag encoding unit that encodes the transform skip flag generated by the flag generation unit and generates encoded data of the transform skip flag; a bitstream generation unit that generates a bitstream including encoded data of the transform skip flag generated by the flag encoding unit; An image processing device comprising: (2) a mode control unit that controls, based on the transform skip flag corresponding to the component identifier generated by the flag generation unit, whether a coding mode of coefficient data corresponding to the component identifier is a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process; and a coefficient data encoding unit that encodes coefficient data corresponding to the component identifier in the encoding mode set by the mode control unit and generates encoded data of the coefficient data; Furthermore, The bit stream generation unit generates a bit stream including coded data of the transform skip flag generated by the flag coding unit and coded data of the coefficient data generated by the coefficient data coding unit. The image processing device according to (1). (3) When the conversion process is skipped, the coefficient data encoding unit shares a context variable between encoding a luminance component and encoding a chrominance component of the coefficient data. (2) An image processing device according to the present invention. (4) The coefficient data encoding unit applies a sign code encoding method according to the transform skip flag generated by the flag generation unit. An image processing device according to (2) or (3). (5) The flag generation unit further generates a transform skip residual coding use flag, which is flag information indicating whether to apply a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process; and the flag encoding unit further encodes the transform skip residual coding use flag generated by the flag generation unit to generate encoded data of the transform skip residual coding use flag; The bitstream generation unit generates the bitstream further including encoded data of the transform skip residual coding use flag generated by the flag encoding unit. An image processing device according to any one of (1) to (4). (6) The flag generation unit further generates a transform skip residual coding use specific mode flag, which is flag information indicating whether to apply a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process, in a specific mode; the flag encoding unit further encodes the transform skip residual encoding use specific mode flag generated by the flag generation unit to generate encoded data of the transform skip residual encoding use specific mode flag; The bitstream generation unit generates the bitstream further including encoded data of the transform skip residual encoding use specific mode flag generated by the flag encoding unit. An image processing device according to any one of (1) to (5). (7) In encoding an image, a transform skip flag is generated, which is flag information indicating, for each component, whether or not to skip a transform process for transforming a residual between the image and a predicted image of the image into coefficient data; encoding the generated transform skip flag to generate encoded data of the transform skip flag; generating a bitstream including encoded data of the generated transform skip flag; Bitstream generation method.

[0316] (8) In encoding an image, when a transform skip flag corresponding to a component identifier indicates a transform skip that skips a transform process that converts a residual between the image and a predicted image of the image into a transform coefficient, a quantization parameter correction unit that corrects a quantization parameter to be applied to a transform block to be processed that corresponds to the component identifier; a quantization unit that quantizes a target transform block corresponding to the component identifier by using the quantization parameter corrected by the quantization parameter correction unit; An image processing device comprising: (9) The quantization parameter correction unit When the transform skip flag corresponding to the component identifier indicates the transform skip, a larger value of the minimum quantization parameter of the transform skip and the quantization parameter corresponding to the component identifier is set as a quantization parameter to be applied to the transform block to be processed corresponding to the component identifier; When the transform skip flag corresponding to the component identifier indicates a non-transform skip in which the transform skip is not performed, the quantization parameter corresponding to the component identifier is set as the quantization parameter to be applied to the transform block to be processed corresponding to the component identifier. (8) An image processing device according to (8). (10) In encoding an image, when a transform skip flag corresponding to a component identifier indicates a transform skip for skipping a transform process for converting a residual between the image and a predicted image of the image into a transform coefficient, a quantization parameter to be applied to a transform block to be processed corresponding to the component identifier is corrected; Using the corrected quantization parameter, the transform block to be processed corresponding to the component identifier is quantized to generate quantized coefficients corresponding to the component identifier. Quantization coefficient generation method.

[0317] (11) a flag decoding unit that decodes encoded data of a transform skip flag corresponding to a component identifier and obtains the transform skip flag corresponding to the component identifier; a mode control unit that controls, based on the transform skip flag corresponding to the component identifier obtained by the flag decoding unit, whether a decoding mode of coded data of coefficient data corresponding to the component identifier is set to a TS residual decoding mode, which is a mode for skipping an inverse transform process of transforming coefficient data into a residual between an image and a predicted image, or a non-TS residual decoding mode, which is a mode for not skipping the inverse transform process; a coefficient data decoding unit that decodes the coded data of the coefficient data corresponding to the component identifier in accordance with the decoding mode set by the mode control unit, and generates the coefficient data corresponding to the component identifier; An image processing device comprising: (12) When the inverse transform process is skipped, the coefficient data decoding unit shares a context variable between decoding the coded data of the luminance component of the coefficient data and decoding the coded data of the chrominance component of the coefficient data. The image processing device according to (11). (13) The coefficient data decoding unit applies a sign code decoding method according to the transform skip flag corresponding to the component identifier obtained by the flag decoding unit. The image processing device according to (11) or (12). (14) The flag decoding unit further includes a flag decoding unit that decodes encoded data of a transform skip residual coding use flag, which is flag information indicating whether to apply the TS residual decoding mode or the non-TS residual decoding mode when the inverse transform process is skipped, and obtains the transform skip residual coding use flag corresponding to the component identifier. The mode control unit further controls, based on the transform skip residual coding use flag corresponding to the component identifier generated by the flag decoding unit, whether the decoding mode of the encoded data of the coefficient data corresponding to the component identifier is the TS residual decoding mode or the non-TS residual decoding mode. An image processing device according to any one of (11) to (13). (15) The flag decoding unit further decodes encoded data of a transform skip residual coding use specific mode flag, which is flag information indicating whether the TS residual decoding mode or the non-TS residual decoding mode is applied in a specific mode, to obtain the transform skip residual coding use specific mode flag corresponding to the component identifier, The mode control unit further controls, based on the transform skip residual coding use specific mode flag corresponding to the component identifier generated by the flag decoding unit, whether the decoding mode of the coded data of the coefficient data corresponding to the component identifier is the TS residual decoding mode or the non-TS residual decoding mode. An image processing device according to any one of (11) to (14). (16) decoding encoded data of a transform skip flag corresponding to a component identifier to obtain the transform skip flag corresponding to the component identifier; based on the transform skip flag corresponding to the obtained component identifier, controlling whether a decoding mode of the coded data of the coefficient data corresponding to the component identifier is set to a TS residual decoding mode, which is a mode for skipping an inverse transform process of transforming the coefficient data into a residual between an image and a predicted image, or a non-TS residual decoding mode, which is a mode for not skipping the inverse transform process; The coded data of the coefficient data corresponding to the component identifier is decoded in the set decoding mode to generate the coefficient data corresponding to the component identifier. Coefficient data generation method.

[0318] (17) A quantization parameter correction unit that corrects a quantization parameter to be applied to a target transform block corresponding to a component identifier when a transform skip flag corresponding to the component identifier indicates a transform skip that skips an inverse transform process that transforms coefficient data into a residual between an image and a predicted image; an inverse quantization unit that performs inverse quantization on a transform block to be processed corresponding to the component identifier, using the quantization parameter corrected by the quantization parameter correction unit; An image processing device comprising: (18) The quantization parameter correction unit When the transform skip flag corresponding to the component identifier indicates the transform skip, a larger value of the minimum quantization parameter of the transform skip and the quantization parameter corresponding to the component identifier is set as the quantization parameter to be applied to the transform block to be processed corresponding to the component identifier; When the transform skip flag corresponding to the component identifier indicates a non-transform skip in which the transform skip is not performed, the quantization parameter corresponding to the component identifier is set as the quantization parameter to be applied to the transform block to be processed corresponding to the component identifier. (17) An image processing device according to (17). (19) A flag decoding unit is further provided that decodes encoded data of the transform skip flag corresponding to the component identifier to obtain the transform skip flag corresponding to the component identifier, The quantization parameter correction unit corrects the quantization parameter to be applied to the target transform block corresponding to the component identifier when the transform skip flag corresponding to the component identifier obtained by the flag decoding unit indicates the transform skip. The image processing device according to (17) or (18). (20) When a transform skip flag corresponding to a component identifier indicates a transform skip that skips an inverse transform process that transforms coefficient data into a residual between an image and a predicted image, correcting a quantization parameter to be applied to a transform block to be processed that corresponds to the component identifier; Using the corrected quantization parameter, the transform block to be processed corresponding to the component identifier is inverse-quantized to generate the coefficient data corresponding to the component identifier. Coefficient data generation method. [Explanation of symbols]

[0319] 100 image encoding device, 101 control unit, 114 quantization unit, 115 encoding unit, 150 transform mode information encoding unit, 151 control unit, 152 selection unit, 153 TS residual encoding unit, 154 non-TS residual encoding unit, 155 selection unit, 200 image decoding device, 212 decoding unit, 213 inverse quantization unit, 250 transform mode information decoding unit, 251 control unit, 252 selection unit, 253 TS residual decoding unit, 254 non-TS residual decoding unit, 255 selection unit, 311 QP correction unit, 312 quantization processing unit, 411 QP correction unit, 412 inverse quantization processing unit

Claims

1. A bitstream configured to be processed by one or more processing devices, the bitstream including encoded data of a transform skip flag, which is flag information indicating, for each component, whether or not to skip a transform process for transforming a residual between an image and a predicted image of the image into coefficient data, in image encoding. A non-transitory computer-readable medium for storing the

2. The bitstream further includes coded data of coefficient data corresponding to the component identifier, coded in a coding mode set based on the transform skip flag corresponding to the component identifier, out of a TS residual coding mode which is a mode for skipping the transform process and a non-TS residual coding mode which is a mode for not skipping the transform process. The non-transitory computer-readable medium of claim 1 .

3. When the conversion process is skipped, the coded data of the coefficient data is coded by sharing a context variable between coding of a luminance component and coding of a chrominance component of the coefficient data. The non-transitory computer-readable medium of claim 2 .

4. The coded data of the coefficient data is coded by applying a sign code coding method according to the transform skip flag. The non-transitory computer-readable medium of claim 2 .

5. The bitstream further includes encoded data of a transform skip residual coding use flag, which is flag information indicating whether to apply a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process. The non-transitory computer-readable medium of claim 1 .

6. The bitstream further includes encoded data of a transform skip residual coding use specific mode flag, which is flag information indicating whether to apply a TS residual coding mode, which is a mode for skipping the transform process, or a non-TS residual coding mode, which is a mode for not skipping the transform process, in a specific mode. The non-transitory computer-readable medium of claim 1 .

Images