Image processing device and method

Abstract

The present disclosure relates to an image processing device and a method configured to enable the use of an encoding tool for when bit depth is extended, while maintaining compatibility with specifications in which the bit depth of an image is not extended. An operation range extension flag is encoded in accordance with the bit depth of an image, and operation range extension tool control information is encoded in accordance with the operation range extension flag. Alternatively, the operation range extension flag is encoded, and the operation range extension tool control information is encoded in accordance with the bit depth of the image. The present disclosure may be applied to, for example, an image processing device, an image coding device, an image decoding device, a transmitting device, a receiving device, a transceiver device, an information processing device, an imaging device, a reproduction device, an electronic apparatus, an image processing method, an information processing method, and the like.

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method that enable the use of encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. [Background technology]

[0002] Conventionally, coding methods have been proposed in which a prediction residual of a video image is derived, coefficient transformed, quantized, and coded (see, for example, Non-Patent Document 1 and Non-Patent Document 2). It has also been proposed to accommodate an extension of the bit depth of the image to be processed by extending the operation range (also referred to as dynamic range) in such image coding (see, for example, Non-Patent Document 3). Coding tools (operation range extension tools) have also been proposed as functions for high bit depths and high bit rates (see, for example, Non-Patent Documents 4 to 6). By applying the method described in Non-Patent Document 3, the operation range (dynamic range) of an encoder or decoder conforming to the coding / decoding methods described in Non-Patent Document 1 and Non-Patent Document 2 can be extended, and the operation range extension tools described in Non-Patent Documents 4 to 6 can be used. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, “Versatile Video Coding (Draft 10)”, JVET-T2001-v2, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, version 1 - date 2020-10-27 [Non-patent document 2] Jianle Chen, Yan Ye, Seung Hwan Kim, “Algorithm description for Versatile Video Coding and Test Model 11 (VTM 11)”, JVET-T2002-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, version 1 - date 2020-10-27 [Non-Patent Document 3] Frank Bossen, Benjamin Bross, Tomohiro Ikai, Dmytro Rusanovskyy, Ye-Kui Wang, "VVC operation range extensions (Draft 3)", JVET-V2005-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, version 1- date 2021-05-07 [Non-Patent Document 4] Dmytro Rusanovskyy, Marta Karczewicz, Luong Pham Van, Muhammed Coban, "CE-related: On history-enhanced method of Rice parameter derivation for regular residual coding (RRC) at high bit depths", JVET-V0106, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 2021-04-19 [Non-Patent Document 5] Hong-Jheng Jhu, Xiaoyu Xiu, Yi-Wen Chen, Wei Chen, Che-Wei Kuo, Xianglin Wang, "CE-2.1: Slice based Rice parameter selection for transform skip residual coding", JVET-V0054-r1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 2021-04-29 [Non-patent document 6] Tianyang Zhou, Takeshi Chujoh, Tomohiro Ikai, "CE-3.1 and CE-3.2: Transform coefficients range extension for high bit-depth coding", JVET-V0047, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 2021-04-13 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the operation range extension tools described in Non-Patent Documents 4 to 6 are coding tools for processing images with extended bit depth, and are not used when the operation range is not extended. The method described in Non-Patent Document 3 may generate a bitstream that is incompatible with the image coding methods described in Non-Patent Documents 1 and 2 when the operation range is not extended. Therefore, there is a risk that decoders compliant with the decoding methods described in Non-Patent Documents 1 and 2 may not be able to decode the bitstream.

[0005] The present disclosure has been made in light of this situation, and makes it possible to use encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. [Means for solving the problem]

[0006] An image processing device according to one aspect of the present technology includes an operation range extension flag encoding unit that encodes an operation range extension flag according to the bit depth of an image, and an operation range extension tool control information encoding unit that encodes operation range extension tool control information according to the operation range extension flag, wherein the operation range extension flag is flag information regarding the extension of the operation range of the image, and the operation range extension tool control information is control information of an operation range extension tool that is used when the operation range is extended.

[0007] An image processing method according to one aspect of the present technology encodes an operation range extension flag according to the bit depth of an image, and encodes operation range extension tool control information according to the operation range extension flag, wherein the operation range extension flag is flag information regarding the extension of the operation range of the image, and the operation range extension tool control information is control information of an operation range extension tool used when the operation range is extended.

[0008] Another aspect of the present technology is an image processing device that includes an operation range extension flag encoding unit that encodes an operation range extension flag, and an operation range extension tool control information encoding unit that encodes operation range extension tool control information according to the bit depth of the image, wherein the operation range extension flag is flag information regarding the extension of the operation range of the image, and the operation range extension tool control information is control information of an operation range extension tool that is used when the operation range is extended.

[0009] Another aspect of the image processing method of the present technology is an image processing method that encodes an operation range extension flag and encodes operation range extension tool control information according to the bit depth of the image, wherein the operation range extension flag is flag information regarding extension of the operation range of the image, and the operation range extension tool control information is control information of an operation range extension tool used when the operation range is extended.

[0010] In the image processing device and method according to one aspect of the present technology, an operation range extension flag is coded according to the bit depth of the image, and operation range extension tool control information is coded according to the operation range extension flag.

[0011] In the image processing device and method according to another aspect of the present technology, an operation range extension flag is coded, and operation range extension tool control information is coded according to the bit depth of the image. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 10 is a diagram illustrating an example of syntax. [Figure 2] FIG. 10 is a diagram illustrating an example of semantics. [Figure 3] FIG. 10 is a diagram illustrating an example of syntax. [Figure 4]FIG. 10 is a diagram illustrating an example of semantics. [Figure 5] FIG. 10 is a diagram illustrating a method for controlling operation range extension information. [Figure 6] FIG. 10 is a diagram illustrating an example of semantics. [Figure 7] FIG. 10 is a diagram illustrating an example of semantics. [Figure 8] 1 is a block diagram showing an example of the main configuration of an operation range extension information encoding device. [Figure 9] 10 is a flowchart illustrating an example of the flow of an operation range extension information encoding process. [Figure 10] FIG. 10 is a block diagram illustrating an example of the main configuration of an operation range extension information decoding device. [Figure 11] 10 is a flowchart illustrating an example of the flow of an operation range extension information decoding process. [Figure 12] FIG. 10 is a block diagram illustrating an example of the main configuration of an operation range extension information decoding device. [Figure 13] 10 is a flowchart illustrating an example of the flow of an operation range extension information decoding process. [Figure 14] FIG. 10 is a diagram illustrating an example of semantics. [Figure 15] FIG. 10 is a diagram illustrating an example of semantics. [Figure 16] 1 is a block diagram showing an example of the main configuration of an operation range extension information encoding device. [Figure 17] 10 is a flowchart illustrating an example of the flow of an operation range extension information encoding process. [Figure 18] 10 is a flowchart illustrating an example of the flow of an operation range extension information decoding process. [Figure 19] FIG. 10 is a diagram illustrating an example of syntax. [Figure 20] 10 is a flowchart illustrating an example of the flow of an operation range extension information encoding process. [Figure 21] FIG. 10 is a block diagram illustrating an example of the main configuration of an operation range extension information decoding device. [Figure 22] 10 is a flowchart illustrating an example of the flow of an operation range extension information decoding process. [Figure 23] FIG. 1 is a block diagram illustrating an example of the main configuration of an image encoding device. [Figure 24] FIG. 2 is a block diagram illustrating an example of the main configuration of an encoding unit. [Figure 25] 10 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 26] FIG. 1 is a block diagram illustrating an example of the main configuration of an image decoding device. [Figure 27] FIG. 2 is a block diagram illustrating an example of the main configuration of a decoding unit. [Figure 28] 10 is a flowchart showing an example of the flow of an image decoding process. [Figure 29] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order. 1. Literature that supports technical content and technical terminology 2. High bit depth and high bit rate extensions 3. Control of operation range extension flag 4. Control of operation range extension tool control information values 5. Transmission control of operation range extension tool control information 6. Embodiment (Image Encoding Device) 7. Embodiment (Image Decoding Device) 8. Notes

[0014] <1. Literature supporting technical content and terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing, as well as the contents of other documents referenced in the following non-patent documents.

[0015] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above) Non-patent document 5: (mentioned above) Non-patent document 6: (mentioned above) Non-Patent Document 7: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-patent document 8: Recommendation ITU-T H.265 (02 / 2018) "High efficiency video coding", February 2018

[0016] In other words, the contents of the above-mentioned non-patent documents are also used as a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and meet the support requirements of the claims.

[0017] Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), sub-block, macroblock, tile, or slice, as described in the above-mentioned non-patent document.

[0018] Furthermore, when specifying such block sizes, the block sizes may be specified not only directly but also indirectly. For example, the block sizes may be specified using identification information for identifying the sizes. Furthermore, for example, the block sizes may be specified by the ratio or difference with respect to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. This may reduce the amount of information and improve coding efficiency. Furthermore, the specification of the block sizes may also include specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0019] <2. Extensions for high bit depth and high bit rate> Conventionally, as described in, for example, Non-Patent Document 1 and Non-Patent Document 2, there has been an encoding method in which a prediction residual of a video image is derived, coefficient transformed, quantized, and encoded. Furthermore, as described in Non-Patent Document 3, there has been a method in such image encoding that corresponds to an extension of the bit depth of the image to be processed by extending the operation range (also referred to as dynamic range). Furthermore, as described in, for example, Non-Patent Document 4 to Non-Patent Document 6, there has been an encoding tool (operation range extension tool) that functions for high bit depth and high bit rate. By applying the method described in Non-Patent Document 3, the operation range (dynamic range) of an encoder or decoder that complies with the encoding / decoding method described in Non-Patent Document 1 and Non-Patent Document 2 can be extended, and the operation range extension tool described in Non-Patent Document 4 to Non-Patent Document 6 can be used.

[0020] In this specification, extending the operation range (dynamic range) refers to extending the bit depth of images that an encoder or decoder can process (support). This bit depth may be any number of bits. For example, an encoder or decoder may be capable of processing images with a bit depth of 10 bits or less in its basic specifications, but by extending the operation range, it may become possible to process images with a bit depth greater than 10 bits (e.g., 12 bits). In this case, the operation range extension tool is a tool that can be used when an encoder or decoder processes (encodes or decodes) images with a bit depth greater than 10 bits (images with a high bit depth or high bit rate). The extended operation range is also referred to as an extended dynamic range.

[0021] The operation range extension tools described in Non-Patent Documents 4 to 6 are encoding tools for processing images with extended bit depth, and are not used when the operation range of the encoder or decoder is not extended.

[0022] 1 is a diagram showing an example of syntax related to an operation range extension tool described in Non-Patent Document 3. FIG. 2 is a diagram showing an example of the semantics of each syntax element shown in FIG.

[0023] If sps_extension_flag is true (e.g., "1"), it indicates that the operation range extension flag (sps_range_extension_flag) exists in the sequence parameter set (SPS (Sequence Parameter Set)), etc. If sps_extension_flag is false (e.g., "0"), it indicates that the operation range extension flag (sps_range_extension_flag) does not exist in the sequence parameter set, etc.

[0024] The operation range extension flag (sps_range_extension_flag) is flag information regarding the extension of the operation range of the image. If sps_range_extension_flag is true (e.g., "1"), it indicates that operation range extension tool control information (sps_range_extension()) exists in the sequence parameter set, etc. If sps_range_extension_flag is false (e.g., "0"), it indicates that sps_range_extension() does not exist in the sequence parameter set, etc.

[0025] The operation range extension tool control information (sps_range_extension()) is control information for the operation range extension tool used when the operation range is extended. An example of the syntax of sps_range_extension() is shown in Figure 3. Figure 4 shows an example of the semantics of each syntax element shown in Figure 3.

[0026] As shown in FIG. 3, the operation range extension tool control information (sps_range_extension()) may include, for example, a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. This control flag is flag information that controls whether to apply the encoding tool that extends the internal calculation precision of the orthogonal transform described in Non-Patent Document 6. For example, when extended_precision_processing_flag is true (e.g., "1"), this indicates that the extended dynamic range can be used for the transform coefficients and the transform. On the other hand, when extended_precision_processing_flag is false (e.g., "0"), this indicates that the extended dynamic range cannot be used for the transform coefficients and the transform.

[0027] Furthermore, the operation range extension tool control information (sps_range_extension()) may include, for example, a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC (Transform Skip Residual Coding). This control flag is flag information that controls whether or not to apply the coding tool that extends the Rice parameter of TSRC described in Non-Patent Document 5. For example, when sps_ts_residual_coding_rice_present_in_sh_flag is true (e.g., "1"), it indicates that sh_ts_residual_coding_rice_idx_minus1 may be present in a slice header that references the sequence parameter set. When sps_ts_residual_coding_rice_present_in_sh_flag is false (e.g., "0"), it indicates that sh_ts_residual_coding_rice_idx_minus1 is not present in a slice header that references the sequence parameter set.

[0028] Furthermore, the operation range extension tool control information (sps_range_extension()) may include, for example, control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameters of RRC (Regular Residual Coding). These control flags are flag information that controls the coding tool that extends the RRC Rice parameters described in Non-Patent Document 4. For example, when sps_rrc_rice_extension_flag is true (for example, "1"), this indicates that the extension of the derivation of the Rice parameters for binarization of abs_remaining[] and dec_abs_level[] is enabled. When sps_rrc_rice_extension_flag is false (for example, "0"), this indicates that the extension of the derivation of the Rice parameters is disabled. Also, sps_persistent_rice_adaptation_enabled_flag, when true (e.g., "1"), indicates that the derivation of the Rice parameters for binarization of abs_remainder[] and dec_abs_level[] is initialized at the start of each TU using statistics accumulated from the previous TU. Also, sps_persistent_rice_adaptation_enabled_flag, when false (e.g., "0"), indicates that the derivation of the Rice parameters does not use previous TU state.

[0029] As described above, when the operation range extension flag (sps_range_extension_flag) is true (for example, "1"), the operation range extension tool control information (sps_range_extension()) is coded and transmitted.

[0030] However, such a method may generate a bitstream that is incompatible with the image coding method described in Non-Patent Document 1 and Non-Patent Document 2 when the operation range is not extended. For example, even when the bit depth of the image to be processed is 10 bits or less (i.e., when the operation range is not extended), there is a risk that operation range extension tool control information with a value that enables use of the operation range extension tool (if the operation range extension flag is true) may be coded and transmitted. Such a bitstream may be incompatible with the image coding method described in Non-Patent Document 1 and Non-Patent Document 2, and may not be correctly decoded by a decoder that does not support operation range extension.

[0031] Therefore, the operation range extension information regarding the operation range extension is controlled based on the bit depth, as shown in the top row of the table in Fig. 5. In this specification, the operation range extension information may include, for example, an operation range extension flag and operation range extension tool control information.

[0032] <3. Control of operation range extension flag> <Method 1> For example, as shown in the second row from the top of the table in Fig. 5, the value of the operation range extension flag may be controlled based on the bit depth (Method 1). Fig. 6 is a diagram showing an example of semantics in this case. As shown in a rectangular box 11 in Fig. 6, if the bit depth (BitDepth) of the image is 10 bits or less, a bitstream constraint is set such that the value of sps_range_extension_flag is equal to false (e.g., "0").

[0033] By observing such constraints, (the encoded data of) operation range extension tool control information is not stored in a bitstream when the operation range is not extended. Therefore, this bitstream can maintain compatibility with the image coding methods described in Non-Patent Document 1 and Non-Patent Document 2. Therefore, even a decoder that does not support operation range extension can correctly decode this bitstream. In other words, it is possible to use a coding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0034] Fig. 7 is a diagram showing an example of semantics in this case. As shown in a rectangular box 12 in Fig. 7, an internal variable (ExtendedPrecisionFlag) related to the extension of the internal arithmetic precision of the orthogonal transform may be set to the value of a control flag (extended_precision_processing_flag) related to the extension of the internal arithmetic precision of the orthogonal transform. In other words, the value of the internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag). In other words, in this case, control based on the bit depth (BitDepth) of the image is not required for setting the value of the internal variable (ExtendedPrecisionFlag).

[0035] <Operation range extension information encoding device> Fig. 8 is a block diagram showing an example of the configuration of an operation range extension information encoding device, which is one aspect of an image processing device to which the present technology is applied. The operation range extension information encoding device 100 shown in Fig. 8 is a device that performs processing related to encoding of operation range extension information. For example, the operation range extension information encoding device 100 acquires operation range extension information such as an operation range extension flag and operation range extension tool control information. The operation range extension information encoding device 100 also acquires a parameter BitDepth that indicates the bit depth of the image. The operation range extension information encoding device 100 uses this information to perform processing related to encoding of the operation range extension information.

[0036] The operation range extension information encoding device 100 appropriately outputs encoded data of the operation range extension information as a processing result. This encoded data is supplied to, for example, an encoder that encodes an image. The encoder encodes the image based on the operation range extension information and generates a bitstream including the encoded data of the image. The encoded data of the operation range extension information is supplied to the decoding side, for example, by being stored in the bitstream.

[0037] Note that Fig. 8 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 8. In other words, the operation range extension information encoding device 100 may have processing units that are not shown as blocks in Fig. 8. Furthermore, the operation range extension information encoding device 100 may have processing and data flows that are not shown as arrows, etc. in Fig. 8.

[0038] As shown in FIG. 8, the operation range extension information encoding device 100 includes an operation range extension flag encoding unit 111 and an operation range extension tool control information encoding unit 112.

[0039] The operation range extension flag encoding unit 111 performs processing related to encoding of the operation range extension flag (sps_range_extension_flag). For example, the operation range extension flag encoding unit 111 may acquire the operation range extension flag (sps_range_extension_flag). The operation range extension flag encoding unit 111 may also acquire a parameter (BitDepth) indicating the bit depth of the image (target of encoding).

[0040] The operation range extension flag encoding unit 111 encodes the operation range extension flag (sps_range_extension_flag) according to the bit depth (BitDepth) of the image. For example, if the bit depth of the image is not to be extended, the operation range extension flag encoding unit 111 may set the operation range extension flag to false and perform encoding, and if the bit depth of the image is to be extended, the operation range extension flag encoding unit 111 may set the operation range extension flag to false and perform encoding, if the bit depth of the image is 10 bits or less, and may encode the operation range extension flag if the bit depth of the image is greater than 10 bits.

[0041] The operation range extension flag encoding unit 111 may output, as a processing result, encoded data of the operation range extension flag to the outside of the operation range extension information encoding device 100. In addition, the operation range extension flag encoding unit 111 may supply the operation range extension flag (sps_range_extension_flag) to the operation range extension tool control information encoding unit 112.

[0042] The operation range extension tool control information encoder 112 performs processing related to encoding of the operation range extension tool control information (sps_range_extension()). For example, the operation range extension tool control information encoder 112 may acquire the operation range extension tool control information (sps_range_extension()). Alternatively, the operation range extension tool control information encoder 112 may acquire the operation range extension flag (sps_range_extension_flag) supplied from the operation range extension flag encoder 111.

[0043] The operation range extension tool control information encoder 112 encodes the operation range extension tool control information in accordance with the operation range extension flag. For example, if the operation range extension flag is false, the operation range extension tool control information encoder 112 may skip (omit) encoding of the operation range extension tool control information. Alternatively, the operation range extension tool control information encoder 112 may set the operation range extension tool control information whose encoding has been skipped to a value indicating that the operation range extension tool is unusable. Alternatively, if the operation range extension flag is true, the operation range extension tool control information encoder 112 may encode the operation range extension tool control information.

[0044] Note that this operation range extension tool control information (sps_range_extension()) may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In this case, the operation range extension tool control information encoding unit 112 may set an internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In other words, as described with reference to FIG. 7, the value of this internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag).

[0045] The operation range extension tool control information (sps_range_extension()) may also include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC. The operation range extension tool control information (sps_range_extension()) may also include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of RRC (Regular Residual Coding).

[0046] The operation range extension tool control information encoding unit 112 may output the encoded data of the operation range extension tool control information to the outside of the operation range extension information encoding device 100 as a processing result.

[0047] With this configuration, the operation range extension information encoding device 100 can encode the operation range extension information in compliance with the constraints shown in Fig. 6. That is, by supplying the generated encoded data of the operation range extension information to an encoder, the operation range extension information encoding device 100 can generate a bitstream that does not violate the constraints shown in Fig. 6. In other words, the operation range extension information encoding device 100 can generate a bitstream that maintains compatibility with the image encoding methods described in Non-Patent Document 1 and Non-Patent Document 2. That is, by the operation range extension information encoding device 100 encoding the operation range extension information in this manner, the encoder and decoder can use encoding tools for bit-depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0048] <Flow of operation range extension information encoding process> An example of the flow of the operation range extension information encoding process executed by the operation range extension information encoding device 100 will be described with reference to the flowchart of FIG.

[0049] In this operation range extension information encoding process, the operation range extension information encoding device 100 encodes the operation range extension flag according to the bit depth of the image, and encodes the operation range extension tool control information according to the operation range extension flag.

[0050] For example, if the bit depth of the image is not to be extended, the operation range extension flag encoding unit 111 may set the operation range extension flag to false and perform encoding, and if the bit depth of the image is to be extended, the operation range extension flag encoding unit 111 may set the operation range extension flag to false and perform encoding, if the bit depth of the image is 10 bits or less, and may encode the operation range extension flag if the bit depth of the image is greater than 10 bits.

[0051] When the operation range extension information encoding process starts, the operation range extension flag encoding unit 111 determines whether or not the bit depth (BitDepth) of the image is greater than 10 bits in step S101 of FIG.

[0052] If it is determined that the bit depth (BitDepth) of the image is greater than 10 bits, the process proceeds to step S102. In this case, in step S102, the operation range extension flag encoding unit 111 encodes the operation range extension flag (sps_range_extension_flag). When the process of step S102 ends, the process proceeds to step S104.

[0053] Furthermore, if it is determined in step S101 that the bit depth (BitDepth) of the image is 10 bits or less, the process proceeds to step S103. In this case, in step S103, the operation range extension flag encoding unit 111 sets the operation range extension flag (sps_range_extension_flag) to false (for example, "0") and performs encoding (sps_range_extension_flag = 0). When the process of step S103 ends, the process proceeds to step S104.

[0054] For example, if the operation range extension flag is false, the operation range extension tool control information encoder 112 may skip encoding of the operation range extension tool control information. In this case, the operation range extension tool control information encoder 112 may set the operation range extension tool control information whose encoding has been skipped to a value indicating that the operation range extension tool is unusable. Alternatively, if the operation range extension flag is true, the operation range extension tool control information encoder 112 may encode the operation range extension tool control information.

[0055] In step S104, the operation range extension tool control information encoder 112 determines whether the operation range extension flag is true (for example, "1"). If it is determined that the operation range extension flag is true, the process proceeds to step S105. In step S105, the operation range extension tool control information encoder 112 encodes the operation range extension tool control information. When the process of step S105 ends, the operation range extension information encoding process ends.

[0056] Also, if it is determined in step S104 that the operation range extension flag is false, the process proceeds to step S106. In step S106, the operation range extension tool control information encoder 112 skips encoding the operation range extension tool control information. Also, the operation range extension tool control information encoder 112 sets (each control flag of) that operation range extension tool control information to a value indicating that the operation range extension tool is unusable. When the process of step S106 ends, the operation range extension information encoding process ends.

[0057] By performing each process as described above, the operation range extension information encoding device 100 can encode the operation range extension information in compliance with the constraints shown in Fig. 6. In other words, by supplying the generated encoded data of the operation range extension information to an encoder, the operation range extension information encoding device 100 can generate a bitstream that does not violate the constraints shown in Fig. 6. Therefore, the encoder and decoder can use encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0058] <Operation range extension information decoding device> Fig. 10 is a block diagram showing an example of the configuration of an operation range extension information decoding device, which is one aspect of an image processing device to which the present technology is applied. The operation range extension information decoding device 200 shown in Fig. 10 is a device that performs processing related to decoding of coded data of operation range extension information. For example, the operation range extension information decoding device 200 obtains coded data of operation range extension information (such as an operation range extension flag and operation range extension tool control information) generated by the operation range extension information coding device 100.

[0059] This operation range extension information maintains compatibility with the image coding methods described in Non-Patent Document 1 and Non-Patent Document 2. In other words, an image bitstream coded using this operation range extension information can be correctly decoded even by a decoder that does not support the operation range extension.

[0060] The operation range extension information decoding device 200 performs processing related to decoding of the coded data of the operation range extension information. As a result of the processing, the operation range extension information decoding device 200 appropriately outputs operation range extension information. This operation range extension information is supplied to, for example, a decoder that decodes coded data of an image. The decoder decodes the coded data based on the operation range extension information and generates (reconstructs) an image.

[0061] Note that Fig. 10 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, the operation range extension information decoding device 200 may have processing units that are not shown as blocks in Fig. 10. Furthermore, the operation range extension information decoding device 200 may have processing and data flows that are not shown as arrows, etc. in Fig. 10.

[0062] As shown in FIG. 10, the operation range extension information decoder 200 includes an operation range extension flag decoder 211 and an operation range extension tool control information decoder 212 .

[0063] The operation range extension flag decoding unit 211 performs processing related to decoding of coded data of the operation range extension flag (sps_range_extension_flag). For example, the operation range extension flag decoding unit 211 may obtain coded data of the operation range extension flag. The operation range extension flag decoding unit 211 may also decode the coded data and generate (restore) the operation range extension flag. The operation range extension flag decoding unit 211 may also output the operation range extension flag as a processing result to the outside of the operation range extension information decoding device 200. The operation range extension flag decoding unit 211 may also supply the operation range extension flag to the operation range extension tool control information decoding unit 212.

[0064] The operation range extension tool control information decoding unit 212 performs processing related to decoding of the coded data of the operation range extension tool control information (sps_range_extension()). For example, the operation range extension tool control information decoding unit 212 may acquire the coded data of the operation range extension tool control information. Alternatively, the operation range extension tool control information decoding unit 212 may acquire an operation range extension flag supplied from the operation range extension flag decoding unit 211. The operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information according to the operation range extension flag. Alternatively, the operation range extension tool control information decoding unit 212 may output the operation range extension tool control information as a processing result to the outside of the operation range extension information decoding device 200.

[0065] Note that this operation range extension tool control information may include a control flag (extended_precision_processing_flag) related to extension of the internal calculation precision of the orthogonal transform. In this case, an internal variable (ExtendedPrecisionFlag) related to extension of the internal calculation precision of the orthogonal transform in the decoder may be set to the value of the control flag (extended_precision_processing_flag) related to extension of the internal calculation precision of the orthogonal transform. Also, this operation range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to extension of the Rice parameter of the TSRC. Also, this operation range extension tool control information may include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to extension of the Rice parameter of the RRC.

[0066] With this configuration, the operation range extension information decoding device 200 can correctly decode the coded data of the operation range extension information generated by the operation range extension information coding device 100. In other words, the decoder can use coding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0067] <Flow of operation range extension information decoding process> An example of the flow of the operation range extension information decoding process executed by the operation range extension information decoding device 200 will be described with reference to the flowchart of FIG.

[0068] When the operation range extension information decoding process starts, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag) in step S131 of FIG. 11, and generates (restores) the operation range extension flag.

[0069] In step S132, the operation range extension tool control information decoder 212 determines whether the operation range extension flag is true (for example, "1") or not.

[0070] If it is determined that the operation range extension flag is true, the process proceeds to step S133. In step S133, the operation range extension tool control information decoder 212 decodes the coded data of the operation range extension tool control information (sps_range_extension()). When the process of step S133 ends, the operation range extension information decoding process ends.

[0071] Also, if it is determined in step S132 that the operation range extension flag is false (for example, "0"), the process proceeds to step S134. As described with reference to the flowchart in FIG. 9, if the operation range extension flag is false, the operation range extension tool control information is not transmitted (not encoded). Therefore, in step S134, the operation range extension tool control information decoding unit 212 skips (omits) decoding of the operation range extension tool control information. Then, the operation range extension tool control information decoding unit 212 sets (each control flag of) that operation range extension tool control information to a value indicating that the operation range extension tool is unusable. When the process of step S134 ends, the operation range extension information decoding process ends.

[0072] By performing each process as described above, the operation range extension information decoding device 200 can correctly decode the coded data of the operation range extension information generated by the operation range extension information coding device 100. In other words, the decoder can use coding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0073] <Method 1-1> From the decoder's perspective, it is possible that a bitstream that violates the above-mentioned constraints may be input. Therefore, as shown in the third row from the top of the table in FIG. 5, the decoder may check whether the bitstream violates any constraints during decoding (Method 1-1). The decoder may then decode the bitstream only if it does not violate any constraints, and may terminate or interrupt the decoding (and issue a notification, etc.) if it does violate any constraints. This prevents the decoder from decoding an invalid bitstream, enabling it to decode the bitstream more safely (reducing the occurrence of malfunctions, etc.).

[0074] <Another Example of Operation Range Extension Information Decoding Device> 12 is a block diagram showing an example of the main configuration of an operation range extension information decoding device 200 in this case. In this case as well, the operation range extension information decoding device 200 acquires and decodes the coded data of the operation range extension information generated by the operation range extension information coding device 100. The operation range extension information decoding device 200 appropriately outputs operation range extension information as a result of this processing. This operation range extension information is supplied to, for example, a decoder that decodes the coded data of an image. The decoder decodes the coded data based on this operation range extension information and generates (restores) an image.

[0075] As shown in Figure 12, the operation range extension information decoding device 200 in this case has a constraint violation processing unit 213 in addition to the configuration described with reference to Figure 10 (operation range extension flag decoding unit 211 and operation range extension tool control information decoding unit 212).

[0076] In this case, the operation range extension flag decoding unit 211 and the operation range extension tool control information decoding unit 212 each perform the same processing as in the case of Fig. 10. For example, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag). Furthermore, the operation range extension tool control information decoding unit 212 decodes the coded data of the operation range extension tool control information (sps_range_extension()) according to the operation range extension flag. For example, if the operation range extension flag is false, the operation range extension tool control information decoding unit 212 may skip decoding the coded data of the operation range extension tool control information and set the operation range extension tool control information to a value indicating that the operation range extension tool cannot be used.

[0077] However, when the operation range extension flag is true (for example, "1"), the operation range extension tool control information decoding unit 212 may control the constraint violation processing unit 213 to determine whether or not there is a constraint violation in the bitstream. The operation range extension tool control information decoding unit 212 may also acquire the determination result. Furthermore, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information based on the determination result.

[0078] For example, if it is determined that the bitstream does not violate any constraints, the operation range extension tool control information decoder 212 may decode the coded data of the operation range extension tool control information. Alternatively, if it is determined that the bitstream violates any constraints, the operation range extension tool control information decoder 212 may skip (omit) the decoding. In this case, the operation range extension tool control information decoder 212 may terminate or suspend the decoding of the bitstream by the decoder.

[0079] The constraint violation processing unit 213 performs processing related to constraints on the bitstream. For example, the constraint violation processing unit 213 acquires a parameter (BitDepth) indicating the bit depth of the image. The constraint violation processing unit 213 may also check whether there is a constraint violation in the bitstream under the control of the operation range extension tool control information decoding unit 212. In other words, the constraint violation processing unit 213 can also be considered a checking unit.

[0080] For example, when the operation range extension flag is true, the constraint violation processing unit 213 checks whether there is a constraint violation in the bitstream. To check whether there is a constraint violation in the bitstream, the constraint violation processing unit 213 may check whether the bit depth of the image in the bitstream has been extended. For example, the constraint violation processing unit 213 may check whether the bit depth of the image in the bitstream is greater than 10 bits.

[0081] The constraint violation processing unit 213 may supply the confirmation result obtained as described above (the determination result as to whether or not there is a constraint violation in the bitstream) to the operation range extension tool control information decoding unit 212. Furthermore, if it is determined that there is a constraint violation in the bitstream, the constraint violation processing unit 213 may notify that the bitstream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify the user, an application, or the like of that fact.

[0082] In this case, the operation range extension tool control information (sps_range_extension()) may also include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In this case, the operation range extension tool control information decoder 212 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In other words, as described with reference to FIG. 7, the value of this internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag).

[0083] The operation range extension tool control information (sps_range_extension()) may also include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC. The operation range extension tool control information (sps_range_extension()) may also include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of RRC (Regular Residual Coding).

[0084] With this configuration, the operation range extension information decoding device 200 can prevent the decoding of an unauthorized bitstream, thereby enabling the decoder to decode the bitstream more safely (reducing the occurrence of failures, etc.).

[0085] <Another example of the operation range extension information decoding process flow> An example of the flow of the operation range extension information decoding process executed by the operation range extension information decoding device 200 shown in FIG. 12 will be described with reference to the flowchart of FIG.

[0086] In this operation range extension information decoding process, the operation range extension information decoding device 200 decodes the coded data of the operation range extension flag, and decodes the coded data of the operation range extension tool control information according to the operation range extension flag.

[0087] When the operation range extension information decoding process starts, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag) in step S161 of FIG.

[0088] In step S162, the operation range extension tool control information decoder 212 determines whether the operation range extension flag is true (for example, "1") If it is determined that the operation range extension flag is true, the process proceeds to step S163.

[0089] If the operation range extension flag is true, the constraint violation processing unit 213 may check whether there is a constraint violation in the bitstream. For example, the constraint violation processing unit 213 may check whether the bit depth of the image in the bitstream has been extended. For example, the constraint violation processing unit 213 may check whether the bit depth of the image in the bitstream is greater than 10 bits. For example, in step S163, the constraint violation processing unit 213 determines whether the bit depth of the image is greater than 10 bits. If it is determined that the bit depth of the image is greater than 10 bits, the processing proceeds to step S164.

[0090] The operation range extension flag being true and the image bit depth being greater than 10 bits does not violate the constraints described with reference to FIG. 6. When the bitstream does not violate the constraints in this way, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information. For example, in step S164, the operation range extension tool control information decoding unit 212 decodes the coded data of the operation range extension tool control information to generate (restore) the operation range extension tool control information. When the processing of step S164 ends, the operation range extension information decoding process ends.

[0091] Also, if it is determined in step S163 that the bit depth of the image is 10 bits or less, the process proceeds to step S165. The operation range extension flag being true and the image bit depth being 10 bits or less violates the constraint described with reference to FIG. 6. When the bit stream violates the constraint in this way, the constraint violation processing unit 213 may notify the user (or an application, etc.) that the bit stream does not satisfy the constraint. When the process of step S165 ends, the operation range extension information decoding process ends.

[0092] Also, if it is determined in step S162 that the operation range extension flag is false, the process proceeds to step S166. In this case, the operation range extension tool control information decoder 212 may skip decoding the coded data of the operation range extension tool control information and set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, in step S166, the operation range extension tool control information decoder 212 omits decoding the coded data of the operation range extension tool control information and sets each control flag in the operation range extension tool control information to a value indicating that the tool is unusable. When the process of step S166 ends, the operation range extension information decoding process ends.

[0093] By performing each process in this manner, the operation range extension information decoding device 200 can prevent the decoding of an unauthorized bitstream, thereby enabling the decoder to decode the bitstream more safely (reducing the occurrence of failures, etc.).

[0094] <4. Controlling the value of operation range extension tool control information> <Method 2> For example, as shown in the fourth row from the top of the table in FIG. 5, the value of the operation range extension tool control information may be controlled based on the bit depth (Method 2). FIGS. 14 and 15 are diagrams showing examples of semantics in this case. As shown in a rectangular box 251 in FIG. 14, if the bit depth (BitDepth) of the image is 10 bits or less, a bitstream constraint is imposed that the value of extended_precision_processing_flag is set to false (e.g., "0"). Also, as shown in a rectangular box 253 in FIG. 14, if the bit depth (BitDepth) of the image is 10 bits or less, a bitstream constraint is imposed that the value of sps_ts_residual_coding_rice_present_in_sh_flag is set to false (e.g., "0"). Furthermore, as shown in a rectangular box 254 in Fig. 15, when the bit depth (BitDepth) of an image is 10 bits or less, a bitstream constraint is set such that the value of sps_rrc_rice_extension_flag is equal to false (e.g., "0") Furthermore, as shown in a rectangular box 255 in Fig. 15, when the bit depth (BitDepth) of an image is 10 bits or less, a bitstream constraint is set such that the value of sps_persistent_rice_adaptation_enabled_flag is equal to false (e.g., "0").

[0095] By observing such constraints, (the encoded data of) operation range extension tool control information is not stored in a bitstream when the operation range is not extended. Therefore, this bitstream can maintain compatibility with the image coding methods described in Non-Patent Document 1 and Non-Patent Document 2. Therefore, even a decoder that does not support operation range extension can correctly decode this bitstream. In other words, it is possible to use a coding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0096] 14, an internal variable (ExtendedPrecisionFlag) related to the extension of the internal arithmetic precision of the orthogonal transform may be set to the value of a control flag (extended_precision_processing_flag) related to the extension of the internal arithmetic precision of the orthogonal transform. That is, the value of the internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag). In other words, in this case, it is not necessary to control the setting of the value of the internal variable (ExtendedPrecisionFlag) based on the bit depth (BitDepth) of the image.

[0097] <Operation range extension information encoding device> 16 is a block diagram showing an example of the main configuration of the operation range extension information encoding device 100 in this case. In this case as well, the operation range extension information encoding device 100 is a device that performs processing related to encoding of the operation range extension information. For example, the operation range extension information encoding device 100 acquires operation range extension information such as an operation range extension flag and operation range extension tool control information. The operation range extension information encoding device 100 also acquires a parameter BitDepth that indicates the bit depth of the image. The operation range extension information encoding device 100 uses this information to perform processing related to encoding of the operation range extension information.

[0098] The operation range extension information encoding device 100 appropriately outputs encoded data of the operation range extension information as a processing result. This encoded data is supplied to, for example, an encoder that encodes an image. The encoder encodes the image based on the operation range extension information and generates a bitstream including the encoded data of the image. The encoded data of the operation range extension information is supplied to the decoding side, for example, by being stored in the bitstream.

[0099] Note that Fig. 16 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 16. In other words, the operation range extension information encoding device 100 may have processing units that are not shown as blocks in Fig. 16. Furthermore, the operation range extension information encoding device 100 may have processing and data flows that are not shown as arrows, etc. in Fig. 16.

[0100] 16, the operation range extension information encoding device 100 in this case has the same configuration as that shown in Fig. 8 (the operation range extension flag encoding unit 111 and the operation range extension tool control information encoding unit 112). However, a parameter (BitDepth) indicating the bit depth of the image is supplied to the operation range extension tool control information encoding unit 112.

[0101] In this case, the operation range extension flag encoding unit 111 also performs processing related to encoding of the operation range extension flag (sps_range_extension_flag). For example, the operation range extension flag encoding unit 111 may acquire an operation range extension flag. However, the operation range extension flag encoding unit 111 encodes the operation range extension flag regardless of the bit depth of the image. Furthermore, the operation range extension flag encoding unit 111 may output encoded data of the operation range extension flag as a processing result to the outside of the operation range extension information encoding device 100. Furthermore, the operation range extension flag encoding unit 111 may supply the operation range extension flag to the operation range extension tool control information encoding unit 112.

[0102] In this case, the operation range extension tool control information encoder 112 also performs processing related to encoding of the operation range extension tool control information (sps_range_extension()). For example, the operation range extension tool control information encoder 112 may acquire operation range extension tool control information. The operation range extension tool control information encoder 112 may also acquire an operation range extension flag supplied from the operation range extension flag encoder 111. Furthermore, the operation range extension tool control information encoder 112 may acquire a parameter (BitDepth) indicating the bit depth of the image.

[0103] The operation range extension tool control information encoding unit 112 encodes the operation range extension tool control information according to the bit depth of the image (and the operation range extension flag). For example, if the operation range extension flag is true and the bit depth is not to be extended, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable. Alternatively, if the operation range extension flag is true and the bit depth is to be extended, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable. Alternatively, if the operation range extension flag is true and the bit depth of the image is 10 bits or less, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable. Alternatively, if the operation range extension flag is true and the bit depth of the image is greater than 10 bits, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information.

[0104] Furthermore, if the operation range extension flag is false, the operation range extension tool control information encoder 112 may skip encoding the operation range extension tool control information. In this case, the operation range extension tool control information encoder 112 may set the operation range extension tool control information to a value indicating that the operation range extension tool cannot be used.

[0105] Note that this operation range extension tool control information (sps_range_extension()) may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In this case, the operation range extension tool control information encoding unit 112 may set an internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In other words, as described with reference to FIG. 14, the value of this internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag).

[0106] The operation range extension tool control information (sps_range_extension()) may also include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC. The operation range extension tool control information (sps_range_extension()) may also include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of RRC (Regular Residual Coding).

[0107] The operation range extension tool control information encoding unit 112 may output the encoded data of the operation range extension tool control information to the outside of the operation range extension information encoding device 100 as a processing result.

[0108] With this configuration, the operation range extension information encoding device 100 can encode the operation range extension information in compliance with the constraints shown in Figures 14 and 15. That is, by supplying the generated encoded data of the operation range extension information to an encoder, the operation range extension information encoding device 100 can generate a bitstream that does not violate the constraints shown in Figures 14 and 15. In other words, the operation range extension information encoding device 100 can generate a bitstream that maintains compatibility with the image encoding methods described in Non-Patent Document 1 and Non-Patent Document 2. That is, by the operation range extension information encoding device 100 encoding the operation range extension information in this manner, an encoder or decoder can use encoding tools for bit-depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0109] <Flow of operation range extension information encoding process> An example of the flow of the operation range extension information encoding process executed by the operation range extension information encoding device 100 of FIG. 16 will be described with reference to the flowchart of FIG.

[0110] In this operation range extension information encoding process, the operation range extension information encoding device 100 encodes the operation range extension flag, and encodes the operation range extension tool control information according to the bit depth of the image.

[0111] When the operation range extension information encoding process starts, the operation range extension flag encoding unit 111 encodes the operation range extension flag (sps_range_extension_flag) in step S201 of FIG.

[0112] In step S202, the operation range extension tool control information encoding unit 112 determines whether the operation range extension flag is true (for example, "1"). If it is determined that the operation range extension flag is true, the process proceeds to step S203.

[0113] In this case, if the bit depth of the image is not to be extended, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable. Alternatively, if the bit depth of the image is to be extended, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information. For example, if the bit depth of the image is 10 bits or less, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable. Alternatively, if the bit depth of the image is greater than 10 bits, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information.

[0114] In step S203, the operation range extension tool control information encoding unit 112 determines whether the bit depth (BitDepth) of the image is greater than 10 bits. If it is determined that the bit depth (BitDepth) of the image is greater than 10 bits, the process proceeds to step S204. In step S204, the operation range extension tool control information encoding unit 112 encodes the operation range extension tool control information. When the process of step S204 ends, the operation range extension information encoding process ends.

[0115] Also, if it is determined in step S203 that the bit depth (BitDepth) of the image is 10 bits or less, the process proceeds to step S205. In step S205, the operation range extension tool control information encoding unit 112 sets (each control flag of) the operation range extension tool control information to a value indicating that the operation range extension tool is unusable, and encodes the information. When the process of step S205 ends, the operation range extension information encoding process ends.

[0116] Also, if it is determined in step S202 that the operation range extension flag is false, the process proceeds to step S206. In step S206, the operation range extension tool control information encoder 112 skips encoding the operation range extension tool control information. Also, the operation range extension tool control information encoder 112 sets (each control flag of) that operation range extension tool control information to a value indicating that the operation range extension tool is unusable. When the process of step S206 ends, the operation range extension information encoding process ends.

[0117] By performing each process as described above, the operation range extension information encoding device 100 can encode the operation range extension information in compliance with the constraints shown in Figures 14 and 15. In other words, by supplying the generated encoded data of the operation range extension information to an encoder, the operation range extension information encoding device 100 can generate a bitstream that does not violate the constraints shown in Figures 14 and 15. Therefore, the encoder and decoder can use encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0118] <Operation range extension information decoding device> The main configuration of the operation range extension information decoding device 200 that acquires and decodes the coded data of the operation range extension information (such as the operation range extension flag and operation range extension tool control information) generated by the operation range extension information coding device 100 in Fig. 16 is the same as the example described with reference to Fig. 10. Therefore, the description thereof will be omitted.

[0119] <Flow of operation range extension information decoding process> The operation range extension information decoding process executed by the operation range extension information decoding device 200 follows the same flow as the example described with reference to the flowchart in Fig. 11. Therefore, a description thereof will be omitted.

[0120] <Method 2-1> From the decoder's perspective, it is possible that a bitstream that violates the above-mentioned constraints may be input. Therefore, as shown in the fifth row from the top of the table in FIG. 5, the decoder may check whether the bitstream violates any constraints during decoding (method 2-1). The decoder may then decode the bitstream only if it does not violate any constraints, and may terminate or interrupt the decoding (and issue a notification, etc.) if it does violate any constraints. This prevents the decoder from decoding an invalid bitstream, enabling it to decode the bitstream more safely (reducing the occurrence of malfunctions, etc.).

[0121] <Another Example of Operation Range Extension Information Decoding Device> In this case, the operation range extension information decoding device 200 has the same configuration as the example described with reference to Fig. 12. That is, the operation range extension information decoding device 200 has an operation range extension flag decoding unit 211, an operation range extension tool control information decoding unit 212, and a constraint violation processing unit 213.

[0122] In this case, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag). Furthermore, the operation range extension tool control information decoding unit 212 decodes the coded data of the operation range extension tool control information (sps_range_extension()) according to the operation range extension flag. For example, if the operation range extension flag is false, the operation range extension tool control information decoding unit 212 may skip decoding the coded data of the operation range extension tool control information and set the operation range extension tool control information to a value indicating that the operation range extension tool cannot be used.

[0123] On the other hand, if the operation range extension flag is true, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information. Then, the operation range extension tool control information decoding unit 212 may control the constraint violation processing unit 213 to determine whether or not there is a constraint violation in the bitstream. That is, the constraint violation processing unit 213 may check whether or not there is a constraint violation in the bitstream.

[0124] For example, the constraint violation processing unit 213 may check whether the bit depth of the image in the bit stream has not been extended and whether the operation range extension tool control information is a value indicating that the operation range extension tool is unavailable. For example, the constraint violation processing unit 213 may check whether the bit depth in the bit stream is 10 bits or less and whether the operation range extension tool control information is a value indicating that the operation range extension tool is unavailable. If a constraint violation is found in the bit stream, the constraint violation processing unit 213 may notify that the bit stream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify a user, an application, or the like of this fact.

[0125] In this case, the operation range extension tool control information (sps_range_extension()) may also include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In this case, the operation range extension tool control information decoder 212 may set the internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In other words, as described with reference to FIG. 7, the value of this internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag).

[0126] The operation range extension tool control information (sps_range_extension()) may also include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC. The operation range extension tool control information (sps_range_extension()) may also include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of RRC (Regular Residual Coding).

[0127] With this configuration, the operation range extension information decoding device 200 can prevent the decoding of an unauthorized bitstream, thereby enabling the decoder to decode the bitstream more safely (reducing the occurrence of failures, etc.).

[0128] <Another example of the operation range extension information decoding process flow> An example of the flow of the operation range extension information decoding process executed by the operation range extension information decoding device 200 (FIG. 12) in this case will be described with reference to the flowchart of FIG.

[0129] In this operation range extension information decoding process, the operation range extension information decoding device 200 decodes the coded data of the operation range extension flag, and decodes the coded data of the operation range extension tool control information according to the operation range extension flag.

[0130] When the operation range extension information decoding process starts, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag) in step S261 of FIG.

[0131] In step S262, the operation range extension tool control information decoding unit 212 determines whether the operation range extension flag is true (for example, "1"). If it is determined that the operation range extension flag is true, the process proceeds to step S263. In step S263, the operation range extension tool control information decoding unit 212 decodes the coded data of the operation range extension tool control information and generates (restores) the operation range extension tool control information.

[0132] In this case, the constraint violation processing unit 213 may use the operation range extension tool control information to check whether there is a constraint violation in the bitstream. For example, the constraint violation processing unit 213 may check whether the bit depth of the image in the bitstream has not been extended and whether the operation range extension tool control information does not have a value indicating that the operation range extension tool is unusable. Alternatively, the constraint violation processing unit 213 may check whether the bit depth in the bitstream is 10 bits or less and whether the operation range extension tool control information does not have a value indicating that the operation range extension tool is unusable. If there is a constraint violation in the bitstream, the constraint violation processing unit 213 may notify that the bitstream does not satisfy the constraint. For example, the constraint violation processing unit 213 may notify a user, an application, or the like of that fact.

[0133] For example, in step S264, the constraint violation processing unit 213 determines whether the bit depth in the bitstream is 10 bits or less and the operation range extension tool control information does not have a value indicating that the operation range extension tool is unusable. If it is determined that the bit depth in the bitstream is 10 bits or less and the operation range extension tool control information does not have a value indicating that the operation range extension tool is unusable, the process proceeds to step S265. In step S265, the constraint violation processing unit 213 notifies the user (or an application, etc.) that the bitstream does not satisfy the constraint. When the process of step S265 ends, the operation range extension information decoding process ends.

[0134] Also, if it is determined in step S264 that the bit depth in the bitstream is greater than 10 bits or the operation range extension tool control information is a value indicating that the operation range extension tool cannot be used, the processing of step S265 is omitted and the operation range extension information decoding processing ends.

[0135] If it is determined in step S262 that the operation range extension flag is false, the process proceeds to step S266. In this case, the operation range extension tool control information decoding unit 212 may skip decoding the coded data of the operation range extension tool control information and set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, in step S266, the operation range extension tool control information decoding unit 212 omits decoding the coded data of the operation range extension tool control information and sets each control flag in the operation range extension tool control information to a value indicating that the tool is unusable. When the process of step S266 ends, the operation range extension information decoding process ends.

[0136] By performing each process in this manner, the operation range extension information decoding device 200 can prevent the decoding of an unauthorized bitstream, thereby enabling the decoder to decode the bitstream more safely (reducing the occurrence of failures, etc.).

[0137] <5. Transmission control of operation range extension tool control information> <Method 3> For example, as shown in the bottom row of the table in FIG. 5, transmission of operation range extension tool control information may be controlled based on bit depth (Method 3). FIG. 19 is a diagram showing an example of syntax related to operation range extension tool control information in this case. In this case, compared to the example in FIG. 3, the lines shown in gray have been added. That is, an if statement that uses bit depth as a condition has been added. In other words, each control flag in the operation range extension tool control information (sps_range_extension()) is coded and transmitted only if the bit depth of the image is greater than 10 bits.

[0138] By controlling the transmission of the operation range extension tool control information in this manner, (the encoded data of) the operation range extension tool control information is not stored in a bitstream when the operation range is not extended. Therefore, this bitstream can maintain compatibility with the image coding methods described in Non-Patent Document 1 and Non-Patent Document 2. Therefore, even decoders that do not support operation range extension can correctly decode this bitstream. In other words, it is possible to use a coding tool for bit depth extension while maintaining compatibility with specifications that do not extend the image bit depth. Furthermore, by doing so, when the image bit depth is not extended (when the bit depth is 10 bits or less), it is possible to omit transmission of the control flag for the high bit depth / high bit rate coding tool in the parameter set. This can suppress an increase in the amount of code.

[0139] <Operation range extension information encoding device> In this case as well, the operation range extension information encoding device 100 is a device that performs processing related to encoding of the operation range extension information. For example, the operation range extension information encoding device 100 acquires operation range extension information such as an operation range extension flag and operation range extension tool control information. The operation range extension information encoding device 100 also acquires a parameter BitDepth that indicates the bit depth of the image. The operation range extension information encoding device 100 uses this information to perform processing related to encoding of the operation range extension information.

[0140] The operation range extension information encoding device 100 appropriately outputs encoded data of the operation range extension information as a processing result. This encoded data is supplied to, for example, an encoder that encodes an image. The encoder encodes the image based on the operation range extension information and generates a bitstream including the encoded data of the image. The encoded data of the operation range extension information is supplied to the decoding side, for example, by being stored in the bitstream.

[0141] The main configuration of the operation range extension information encoding device 100 in this case is the same as the example described with reference to Fig. 16. That is, the operation range extension information encoding device 100 in this case has an operation range extension flag encoding unit 111 and an operation range extension tool control information encoding unit 112.

[0142] In this case, too, the operation range extension flag encoding unit 111 encodes the operation range extension flag regardless of the bit depth of the image. However, the operation range extension tool control information encoding unit 112 encodes the operation range extension tool control information according to the operation range extension flag and the bit depth of the image. For example, the operation range extension tool control information encoding unit 112 may skip (omit) encoding the operation range extension tool control information when the operation range extension flag is false or the bit depth is not to be extended. Alternatively, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information when the operation range extension flag is true and the bit depth is to be extended. For example, the operation range extension tool control information encoding unit 112 may skip (omit) encoding the operation range extension tool control information when the operation range extension flag is false or the bit depth of the image is 10 bits or less. Furthermore, the operation range extension tool control information encoding unit 112 may encode the operation range extension tool control information when the operation range extension flag is true and the bit depth of the image is greater than 10 bits.

[0143] Furthermore, if the operation range extension flag is false or the bit depth of the image is not to be extended, the operation range extension tool control information encoding unit 112 may further set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. Furthermore, if the operation range extension flag is false or the bit depth of the image is 10 bits or less, the operation range extension tool control information encoding unit 112 may further set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable.

[0144] Note that this operation range extension tool control information (sps_range_extension()) may include a control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In this case, the operation range extension tool control information encoding unit 112 may set an internal variable (ExtendedPrecisionFlag) related to the extension of the internal calculation precision of the orthogonal transform in the encoder to the value of the control flag (extended_precision_processing_flag) related to the extension of the internal calculation precision of the orthogonal transform. In other words, as described with reference to FIG. 14, the value of this internal variable (ExtendedPrecisionFlag) may be set equal to the value of the control flag (extended_precision_processing_flag).

[0145] The operation range extension tool control information (sps_range_extension()) may also include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to the extension of the Rice parameter of TSRC. The operation range extension tool control information (sps_range_extension()) may also include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to the extension of the Rice parameter of RRC (Regular Residual Coding).

[0146] The operation range extension tool control information encoding unit 112 may output the encoded data of the operation range extension tool control information to the outside of the operation range extension information encoding device 100 as a processing result.

[0147] With this configuration, the operation range extension information encoding device 100 can prevent (the encoded data of) the operation range extension tool control information from being stored in a bitstream when the operation range is not extended. Therefore, this bitstream can maintain compatibility with the image encoding methods described in Non-Patent Documents 1 and 2. Therefore, even a decoder that does not support operation range extension can correctly decode this bitstream. Therefore, an encoder or decoder can use an encoding tool for bit-depth extension while maintaining compatibility with specifications that do not extend the image bit depth. Furthermore, by doing so, when the image bit depth is not extended (when the bit depth is 10 bits or less), transmission of the control flag for the high bit-depth / high bit-rate encoding tool in the parameter set can be omitted. This can suppress an increase in the amount of code.

[0148] <Another example of the operation range extension information encoding process flow> An example of the flow of the operation range extension information encoding process executed by the operation range extension information encoding device 100 in this case will be described with reference to the flowchart of FIG.

[0149] In this operation range extension information encoding process, the operation range extension information encoding device 100 encodes the operation range extension flag, and encodes the operation range extension tool control information according to the operation range extension flag and the bit depth of the image.

[0150] When the operation range extension information encoding process starts, the operation range extension flag encoding unit 111 encodes the operation range extension flag (sps_range_extension_flag) in step S301 of FIG.

[0151] The operation range extension tool control information encoder 112 may skip (omit) encoding the operation range extension tool control information if the operation range extension flag is false or the bit depth of the image is not to be extended. Alternatively, the operation range extension tool control information encoder 112 may encode the operation range extension tool control information if the operation range extension flag is true and the bit depth of the image is to be extended. For example, if the operation range extension flag is false or the bit depth of the image is not to be extended, the operation range extension tool control information encoder 112 may further set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, if the operation range extension flag is false or the bit depth is 10 bits or less, the operation range extension tool control information encoder 112 may further set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable.

[0152] In step S302, the operation range extension tool control information encoder 112 determines whether the operation range extension flag is true and the bit depth of the image is greater than 10 bits. If it is determined that the operation range extension flag is true and the bit depth of the image is greater than 10 bits, the process proceeds to step S303. In step S303, the operation range extension tool control information encoder 112 encodes the operation range extension tool control information. When the process of step S303 ends, the operation range extension information encoding process ends.

[0153] Also, if it is determined in step S302 that the operation range extension flag is false or the bit depth of the image is 10 bits or less, the process proceeds to step S304. In step S304, the operation range extension tool control information encoder 112 skips (omits) encoding of the operation range extension tool control information. Also, the operation range extension tool control information encoder 112 sets the operation range extension tool control information to a value indicating that the operation range extension tool cannot be used. For example, when the process of step S303 ends, the operation range extension information encoding process ends.

[0154] By performing each process in this manner, the operation range extension information encoding device 100 can prevent the operation range extension tool control information (encoded data) from being stored in the bitstream when the operation range is not extended. Therefore, the encoder and decoder can use the encoding tool for bit depth extension while maintaining compatibility with specifications that do not extend the image bit depth. Furthermore, by doing so, when the image bit depth is not extended (when the bit depth is 10 bits or less), it is possible to omit transmission of the control flag for the high bit depth / high bit rate encoding tool in the parameter set. Therefore, it is possible to suppress an increase in the amount of code.

[0155] <Operation range extension information decoding device> 21 is a block diagram showing an example of the main configuration of the operation range extension information decoding device 200. In this case as well, the operation range extension information decoding device 200 performs processing related to decoding of the coded data of the operation range extension information.

[0156] For example, the operation range extension information decoding device 200 acquires the coded data of the operation range extension information generated by the operation range extension information coding device 100. In addition, the operation range extension information decoding device 200 acquires a parameter (BitDepth) indicating the bit depth of the image.

[0157] This operation range extension information maintains compatibility with the image coding methods described in Non-Patent Document 1 and Non-Patent Document 2. In other words, an image bitstream coded using this operation range extension information can be correctly decoded even by a decoder that does not support the operation range extension.

[0158] The operation range extension information decoding device 200 decodes the coded data of the operation range extension information using a parameter (BitDepth) indicating the bit depth of the image. The operation range extension information decoding device 200 appropriately outputs operation range extension information as a processing result. This operation range extension information is supplied to, for example, a decoder that decodes the coded data of the image. The decoder decodes the coded data based on the operation range extension information and generates (reconstructs) an image.

[0159] As shown in Fig. 21, the operation range extension information decoding device 200 in this case has the same configuration as that described with reference to Fig. 10. That is, the operation range extension information decoding device 200 has an operation range extension flag decoding unit 211 and an operation range extension tool control information decoding unit 212.

[0160] The operation range extension flag decoding unit 211 performs processing related to decoding of coded data of the operation range extension flag (sps_range_extension_flag). For example, the operation range extension flag decoding unit 211 may obtain coded data of the operation range extension flag. The operation range extension flag decoding unit 211 may also decode the coded data and generate (restore) the operation range extension flag. The operation range extension flag decoding unit 211 may also output the operation range extension flag as a processing result to the outside of the operation range extension information decoding device 200. The operation range extension flag decoding unit 211 may also supply the operation range extension flag to the operation range extension tool control information decoding unit 212.

[0161] The operation range extension tool control information decoding unit 212 performs processing related to decoding of the coded data of the operation range extension tool control information (sps_range_extension()). For example, the operation range extension tool control information decoding unit 212 may acquire the coded data of the operation range extension tool control information. The operation range extension tool control information decoding unit 212 may also acquire the operation range extension flag supplied from the operation range extension flag decoding unit 211. Furthermore, the operation range extension tool control information decoding unit 212 may acquire a parameter (BitDepth) indicating the bit depth of the image.

[0162] The operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information according to the operation range extension flag and the bit depth of the image. For example, if the operation range extension flag is false or the bit depth of the image is not to be extended, the operation range extension tool control information decoding unit 212 may skip (omit) decoding the coded data of the operation range extension tool control information. Alternatively, the operation range extension tool control information decoding unit 212 may set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, if the operation range extension flag is true and the bit depth of the image is to be extended, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information.

[0163] For example, if the operation range extension flag is false or the bit depth is 10 bits or less, the operation range extension tool control information decoding unit 212 may skip (omit) decoding the coded data of the operation range extension tool control information. Also, the operation range extension tool control information decoding unit 212 may set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, if the operation range extension flag is true and the bit depth of the image is greater than 10 bits, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information.

[0164] Furthermore, the operation range extension tool control information decoder 212 may output the operation range extension tool control information to the outside of the operation range extension information decoder 200 as a result of the processing.

[0165] Note that this operation range extension tool control information may include a control flag (extended_precision_processing_flag) related to extension of the internal calculation precision of the orthogonal transform. In this case, an internal variable (ExtendedPrecisionFlag) related to extension of the internal calculation precision of the orthogonal transform in the decoder may be set to the value of the control flag (extended_precision_processing_flag) related to extension of the internal calculation precision of the orthogonal transform. Also, this operation range extension tool control information may include a control flag (sps_ts_residual_coding_rice_present_in_sh_flag) related to extension of the Rice parameter of the TSRC. Also, this operation range extension tool control information may include control flags (sps_rrc_rice_extension_flag, sps_persistent_rice_adaptation_enabled_flag) related to extension of the Rice parameter of the RRC.

[0166] With this configuration, the operation range extension information decoding device 200 can correctly decode the coded data of the operation range extension information generated by the operation range extension information coding device 100. In other words, the decoder can use coding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. In addition, an increase in the amount of code can be suppressed.

[0167] <Flow of operation range extension information decoding process> An example of the flow of the operation range extension information decoding process executed by the operation range extension information decoding device 200 of FIG. 21 will be described with reference to the flowchart of FIG.

[0168] In this operation range extension information decoding process, the operation range extension information decoding device 200 decodes the coded data of the operation range extension flag, and decodes the coded data of the operation range extension tool control information according to the operation range extension flag and the bit depth of the image.

[0169] When the operation range extension information decoding process starts, the operation range extension flag decoding unit 211 decodes the coded data of the operation range extension flag (sps_range_extension_flag) in step S331 of FIG. 22, and generates (restores) the operation range extension flag.

[0170] The operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information according to the operation range extension flag and the bit depth of the image. For example, if the operation range extension flag is false or the bit depth of the image is not to be extended, the operation range extension tool control information decoding unit 212 may skip (omit) decoding the coded data of the operation range extension tool control information. Alternatively, the operation range extension tool control information decoding unit 212 may set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, if the operation range extension flag is true and the bit depth of the image is to be extended, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information.

[0171] For example, if the operation range extension flag is false or the bit depth is 10 bits or less, the operation range extension tool control information decoding unit 212 may skip (omit) decoding the coded data of the operation range extension tool control information. Also, the operation range extension tool control information decoding unit 212 may set the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. For example, if the operation range extension flag is true and the bit depth of the image is greater than 10 bits, the operation range extension tool control information decoding unit 212 may decode the coded data of the operation range extension tool control information.

[0172] In step S332, the operation range extension tool control information decoding unit 212 determines whether the operation range extension flag is true (for example, "1") and the bit depth of the image is greater than 10 bits. If it is determined that the operation range extension flag is true and the bit depth of the image is greater than 10 bits, the process proceeds to step S333.

[0173] In step S333, the operation range extension tool control information decoder 212 decodes the coded data of the operation range extension tool control information to generate (restore) the operation range extension tool control information. When the process of step S333 ends, the operation range extension information decoding process ends.

[0174] On the other hand, if it is determined in step S332 that the operation range extension flag is false or the image bit depth is 10 bits or less, the process proceeds to step S334. In step S334, the operation range extension tool control information decoding unit 212 omits decoding of the coded data of the operation range extension tool control information. The operation range extension tool control information decoding unit 212 then sets each control flag of the operation range extension tool control information to a value indicating that the tool is unusable. When the process of step S334 ends, the operation range extension information decoding process ends.

[0175] By performing each process in this manner, the operation range extension information decoding device 200 can correctly decode the coded data of the operation range extension information generated by the operation range extension information coding device 100. In other words, the decoder can use coding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. In addition, an increase in the amount of code can be suppressed.

[0176] 6. Embodiment (Image Encoding Apparatus) <Image encoding device> The present technology described above can be applied to any configuration. For example, the present technology can be applied to an image coding device. FIG. 23 is a block diagram showing an example of the configuration of an image coding device, which is one aspect of an image processing device to which the present technology is applied. The image coding device 300 shown in FIG. 23 is a device that codes image data of a moving image. For example, the image coding device 300 can code image data of a moving image using an encoding method described in any of the non-patent documents mentioned above.

[0177] Note that Fig. 23 shows the main processing units (blocks), data flows, etc., and does not necessarily show everything. That is, the image coding device 300 may have processing units that are not shown as blocks in Fig. 23. Furthermore, the image coding device 300 may have processes and data flows that are not shown as arrows, etc. in Fig. 23.

[0178] As shown in FIG. 23 , the image coding device 300 includes a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transformation unit 313, a quantization unit 314, a coding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transformation unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0179] <Control unit> The control unit 301 divides the video data held in the rearrangement buffer 311 into blocks (CUs, PUs, transform blocks, etc.) based on an externally or pre-specified block size of the processing unit. The control unit 301 also determines the coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0180] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 301 supplies them to each block. Specifically, this is as follows.

[0181] The header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the encoding unit 315 and the prediction unit 322. The transformation information Tinfo is supplied to the encoding unit 315, the orthogonal transformation unit 313, the quantization unit 314, the inverse quantization unit 317, and the inverse orthogonal transformation unit 318. The filter information Finfo is supplied to the in-loop filter unit 320.

[0182] <Sorting buffer> Each field (input image) of video data is input to the image coding device 300 in its playback order (display order). The reordering buffer 311 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 301, the reordering buffer 311 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 311 supplies each processed input image to the calculation unit 312. The reordering buffer 311 also supplies each input image (original image) to the prediction unit 322 and the in-loop filter unit 320.

[0183] <Arithmetic section> The calculation unit 312 receives an image I corresponding to a block of processing units and a predicted image P supplied from the prediction unit 322, subtracts the predicted image P from the image I as shown in the following equation, derives a prediction residual D, and supplies it to the orthogonal transformation unit 313.

[0184] D = I - P

[0185] <Orthogonal transformation section> The orthogonal transform unit 313 receives as input the prediction residual D supplied from the calculation unit 312 and the transform information Tinfo supplied from the control unit 301, and performs an orthogonal transform on the prediction residual D based on the transform information Tinfo to derive transform coefficients Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate primary transform coefficients, and performs a secondary transform on the primary transform coefficients based on the ST identifier to generate secondary transform coefficients. The orthogonal transform unit 313 supplies the obtained secondary transform coefficients as transform coefficients Coeff to the quantization unit 314. Note that the orthogonal transform unit 313 is not limited to an orthogonal transform and can perform any coefficient transform. That is, the transform coefficients Coeff may be derived by performing any coefficient transform on the prediction residual D. Therefore, the orthogonal transform unit 313 can also be referred to as a coefficient transform unit.

[0186] <Quantization section> The quantization unit 314 receives the transform coefficient Coeff supplied from the orthogonal transformation unit 313 and the transform information Tinfo supplied from the control unit 301 as input, and scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. The quantization rate is controlled by the rate control unit 323. The quantization unit 314 supplies a quantization coefficient level (also referred to as a quantization coefficient qcoeff), which is the level value of the transform coefficient quantized in this way, to the encoding unit 315 and the inverse quantization unit 317.

[0187] <Encoding part> The encoding unit 315 receives as input the quantization coefficient level supplied from the quantization unit 314, various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 301, information on filters such as filter coefficients supplied from the in-loop filter unit 320, and information on the optimal prediction mode supplied from the prediction unit 322. The encoding unit 315 performs variable-length coding (for example, arithmetic coding) on ​​the quantization coefficient level to generate a bit string (encoded data).

[0188] Furthermore, the encoding unit 315 includes information about the filter supplied from the in-loop filter unit 320 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 322 in the prediction mode information Pinfo. Then, the encoding unit 315 encodes the various encoding parameters described above (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to generate a bit string.

[0189] The encoding unit 315 multiplexes the bit strings (encoded data) of the various types of information generated as described above to generate a bit stream of the encoded data. The encoding unit 315 supplies the bit stream to the accumulation buffer 316.

[0190] <Accumulation buffer> The accumulation buffer 316 temporarily stores the bit stream of coded data obtained by the coding unit 315. The accumulation buffer 316 outputs the stored bit stream of coded data to the outside of the image coding device 300 at a predetermined timing. For example, this bit stream is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, etc. In other words, the accumulation buffer 316 also functions as a transmission unit that transmits the bit stream (coded data).

[0191] <Inverse quantization part> The inverse quantization unit 317 performs processing related to inverse quantization. For example, the inverse quantization unit 317 receives the quantization coefficient level supplied from the quantization unit 314 and the transformation information Tinfo supplied from the control unit 301 as input, and scales (inverse quantizes) the value of the quantization coefficient level based on the transformation information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed in the quantization unit 314. The inverse quantization unit 317 supplies the transformation coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.

[0192] <Inverse orthogonal transform section> The inverse orthogonal transform unit 318 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 318 receives as input the transform coefficients Coeff_IQ supplied from the inverse quantization unit 317 and the transform information Tinfo supplied from the control unit 301, and performs inverse orthogonal transform on the transform coefficients Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. This inverse orthogonal transform is the inverse process of the orthogonal transform performed in the orthogonal transform unit 313. The inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 319.

[0193] In other words, the inverse orthogonal transform unit 318 performs the inverse process of the process performed by the orthogonal transform unit 313. That is, similar to the orthogonal transform unit 313, the inverse orthogonal transform unit 318 can perform any inverse coefficient transform, not limited to inverse orthogonal transform. This inverse coefficient transform is the inverse process of the coefficient transform performed by the orthogonal transform unit 313. That is, the prediction residual D' may be derived by performing any inverse coefficient transform on the transform coefficients Coeff_IQ. Therefore, the inverse orthogonal transform unit 318 can also be said to be an inverse coefficient transform unit.

[0194] <Arithmetic section> The calculation unit 319 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 318 and the predicted image P supplied from the prediction unit 322. The calculation unit 319 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to derive a locally decoded image Rlocal. The calculation unit 319 supplies the derived locally decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

[0195] <In-loop filter section> The in-loop filter unit 320 performs processing related to in-loop filtering. For example, the in-loop filter unit 320 receives as input the locally decoded image Rlocal supplied from the calculation unit 319, filter information Finfo supplied from the control unit 301, and an input image (original image) supplied from the rearrangement buffer 311. Note that any information may be input to the in-loop filter unit 320, and information other than the above information may also be input. For example, information such as a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, and a block (CU, CTU, etc.) may be input to the in-loop filter unit 320 as needed.

[0196] The in-loop filter unit 320 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses the input image (original image) and other input information for the filtering, as necessary.

[0197] For example, the in-loop filter unit 320 applies four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order, as described in Non-Patent Document 11. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate.

[0198] Of course, the filtering process performed by the in-loop filter unit 320 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 320 may apply a Wiener filter or the like.

[0199] The in-loop filter unit 320 supplies the filtered locally decoded image Rlocal to the frame memory 321. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 320 supplies the information about the filter to the encoding unit 315.

[0200] <Frame memory> The frame memory 321 performs processing related to the storage of image-related data. For example, the frame memory 321 receives as input the locally decoded image Rlocal supplied from the calculation unit 319 and the filtered locally decoded image Rlocal supplied from the in-loop filter unit 320, and holds (stores) them. The frame memory 321 also reconstructs and holds a decoded image R for each picture using the locally decoded image Rlocal (storing it in a buffer within the frame memory 321). The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.

[0201] <Prediction Department> The prediction unit 322 performs processing related to the generation of a predicted image. For example, the prediction unit 322 receives as input the prediction mode information Pinfo supplied from the control unit 301, the input image (original image) supplied from the rearrangement buffer 311, and the decoded image R (or a portion thereof) read from the frame memory 321. The prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and the input image (original image), performs prediction by referring to the decoded image R as a reference image, and performs motion compensation processing based on the prediction result to generate a predicted image P. The prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319. Furthermore, the prediction unit 322 supplies information on the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 315 as necessary.

[0202] <Rate control section> The rate control unit 323 performs processing related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of the coded data accumulated in the accumulation buffer 316 so as to prevent overflow or underflow.

[0203] These processing units (the control unit 301, the reordering buffer 311, and the rate control unit 323) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Furthermore, each processing unit may have, for example, a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of the processing units may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0204] <Encoding part> FIG. 24 is a block diagram showing an example of the main configuration of the encoding unit 315. As shown in FIG. 24, the encoding unit 315 has an operation range extension information encoding unit 351. The operation range extension information encoding unit 351 performs processing related to encoding of the operation range extension information. For example, the operation range extension information encoding unit 351 may acquire operation range extension information supplied as an encoding parameter from the control unit 301. The operation range extension information encoding unit 351 may also acquire a parameter (BitDepth) indicating the bit depth of the image supplied as an encoding parameter from the control unit 301. The operation range extension information encoding unit 351 may use this information to perform processing related to encoding of the operation range extension information.

[0205] Furthermore, the operation range extension information encoding unit 351 may appropriately output, as a processing result, encoded data of the operation range extension information, etc. This encoded data is stored in the encoding unit 315 in a bitstream including encoded data of the image. This bitstream is output to the outside of the image encoding device 300 via the accumulation buffer 316. This bitstream is supplied to the decoding side. Furthermore, the operation range extension information encoded in the operation range extension information encoding unit 351, the operation range extension information set in the operation range extension information encoding unit 351, or internal variables corresponding thereto are supplied to any processing unit of the image encoding device 300 as encoding parameters and used therein. For example, this operation range extension information (or internal variables) may be supplied to the encoding unit 315 and used for encoding, etc. Alternatively, this operation range extension information (or internal variables) may be supplied to the orthogonal transform unit 313 and used for orthogonal transform, etc. Alternatively, this operation range extension information (or internal variables) may be supplied to the inverse orthogonal transform unit 318 and used for inverse orthogonal transform, etc.

[0206] In the encoding unit 315 configured as above, any of the methods (method 1, method 2, or method 3) of the present technology described above may be applied.

[0207] For example, the operation range extension information encoding unit 351 may have the same configuration as the operation range extension information encoding device 100 shown in Fig. 8 and may execute the operation range extension information encoding process shown in Fig. 9. By doing so, the image encoding device 300 can obtain the effect (the effect when Method 1 is applied) described above in <3. Control of the operation range extension flag>. In other words, the image encoding device 300 can use encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0208] Also, for example, the operation range extension information encoding unit 351 may have the same configuration as the operation range extension information encoding device 100 shown in Fig. 16 and may execute the operation range extension information encoding process shown in Fig. 17. By doing so, the image encoding device 300 can obtain the effect (the effect when Method 2 is applied) described above in <4. Control of the value of the operation range extension tool control information>. In other words, the image encoding device 300 can use an encoding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0209] Also, for example, the operation range extension information encoding unit 351 may have the same configuration as the operation range extension information encoding device 100 shown in Fig. 16 and may execute the operation range extension information encoding process shown in Fig. 20. By doing so, the image encoding device 300 can obtain the effects (effects when Method 3 is applied) described above in <5. Transmission Control of Operation Range Extension Tool Control Information>. In other words, the image encoding device 300 can use an encoding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. Also, an increase in the amount of code can be suppressed.

[0210] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 300 configured as above will be described with reference to the flowchart of FIG.

[0211] When the image encoding process starts, in step S401, the reordering buffer 311 is controlled by the control unit 301 to reorder the frames of the input video data from display order to encoding order.

[0212] In step S402, the control unit 301 sets a processing unit for the input image held in the sorting buffer 311 (performs block division).

[0213] In step S403, the control unit 301 determines (sets) coding parameters for the input image held by the reordering buffer 311.

[0214] In step S404, the prediction unit 322 performs a prediction process to generate a predicted image etc. in an optimal prediction mode. For example, in this prediction process, the prediction unit 322 performs intra prediction to generate a predicted image etc. in an optimal intra prediction mode, performs inter prediction to generate a predicted image etc. in an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value etc.

[0215] In step S405, the calculation unit 312 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the predicted image. The prediction residual D calculated in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to when the image is encoded as is.

[0216] In step S406, the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated in the process of step S405, and derives a transform coefficient Coeff.

[0217] In step S407, the quantization unit 314 quantizes the transform coefficient Coeff obtained by the processing in step S406, for example, by using the quantization parameter calculated by the control unit 301, and derives the quantization coefficient level.

[0218] In step S408, the inverse quantization unit 317 inverse quantizes the quantization coefficient LEVEL generated by the process in step S407 using characteristics corresponding to the quantization characteristics in step S407, and derives the transform coefficient Coeff_IQ.

[0219] In step S409, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process in step S408 using a method corresponding to the orthogonal transform process in step S406, and derives a prediction residual D'.

[0220] In step S410, the calculation unit 319 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S404 to the prediction residual D' derived in the process of step S409.

[0221] In step S411, the in-loop filter unit 320 performs in-loop filtering on the locally decoded image derived in the processing of step S410.

[0222] In step S412, the frame memory 321 stores the locally decoded image derived by the process of step S410 and the locally decoded image filtered in step S411.

[0223] In step S413, the encoding unit 315 encodes the quantization coefficient level and various encoding parameters obtained in the process of step S407, and generates a bit stream of encoded data.

[0224] In step S414, the accumulation buffer 316 accumulates the bitstream obtained in step S413 and outputs it to the outside of the image encoding device 300. This bitstream is transmitted to the decoding side via, for example, a transmission path or a recording medium. In addition, the rate control unit 323 performs rate control as necessary.

[0225] When the process of step S414 ends, the image encoding process ends.

[0226] In step S413 of such image encoding processing, any of the above-described methods (method 1, method 2, or method 3) of the present technology may be applied.

[0227] For example, in step S413, the encoding unit 315 (the operation range extension information encoding unit 351 thereof) may encode the operation range extension information (encoding parameters) by executing the operation range extension information encoding process shown in Fig. 9. In this way, the image encoding device 300 can obtain the effect (the effect when Method 1 is applied) described above in <3. Control of the operation range extension flag>. In other words, the image encoding device 300 can use encoding tools for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0228] Also, for example, in step S413, the encoding unit 315 (the operation range extension information encoding unit 351 thereof) may encode the operation range extension information (encoding parameters) by executing the operation range extension information encoding process shown in Fig. 17. By doing so, the image encoding device 300 can obtain the effect (the effect when Method 2 is applied) described above in <4. Control of the Value of Operation Range Extension Tool Control Information>. In other words, the image encoding device 300 can use an encoding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images.

[0229] Also, for example, in step S413, the encoding unit 315 (the operation range extension information encoding unit 351 thereof) may encode the operation range extension information (encoding parameters) by executing the operation range extension information encoding process shown in Fig. 20. By doing so, the image encoding device 300 can obtain the effect (the effect when Method 3 is applied) described above in <5. Transmission Control of Operation Range Extension Tool Control Information>. In other words, the image encoding device 300 can use an encoding tool for bit depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. Also, an increase in the amount of code can be suppressed.

[0230] 7. Embodiment (Image Decoding Apparatus) <Image decoding device> Fig. 26 is a block diagram showing an example of the configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 400 shown in Fig. 26 is a device that encodes encoded data of a moving image. For example, the image decoding device 400 can decode the encoded data using a decoding method described in any of the above-mentioned non-patent documents. For example, the image decoding device 400 decodes the encoded data (bit stream) generated by the above-mentioned image encoding device 300.

[0231] Note that Figure 26 shows the main processing units (blocks), data flows, etc., and does not necessarily show everything. That is, the image decoding device 400 may have processing units that are not shown as blocks in Figure 26. Furthermore, the image decoding device 400 may have processes and data flows that are not shown as arrows, etc. in Figure 26.

[0232] 26, the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transform unit 414, a calculation unit 415, an in-loop filter unit 416, a rearrangement buffer 417, a frame memory 418, and a prediction unit 419. The prediction unit 419 includes an intra prediction unit and an inter prediction unit (not shown). The image decoding device 400 is a device for generating moving image data by decoding a bit stream including encoded data of moving images. For example, the image decoding device 400 can correctly decode the bit stream generated by the image encoding device 300.

[0233] <Accumulation buffer> The accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400. The accumulation buffer 411 supplies the accumulated bitstream to the decoding unit 412 at a predetermined timing or when a predetermined condition is met.

[0234] <Decryption section> The decoding unit 412 performs processing related to image decoding. For example, the decoding unit 412 receives the bit stream supplied from the accumulation buffer 411 as input, and variable-length decodes the syntax values ​​of each syntax element from the bit string in accordance with the definitions in the syntax table, thereby deriving parameters.

[0235] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc. That is, the decoding unit 412 parses (analyzes and obtains) this information from the bitstream. These pieces of information will be described below.

[0236] <Header information Hinfo> The header information Hinfo includes header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header (SH), etc. The header information Hinfo includes information that specifies, for example, an image size (horizontal width PicWidth, vertical width PicHeight), a bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum CU size (MaxCUSize) / a minimum CU size (MinCUSize), a maximum depth MaxQTDepth / a minimum depth MinQTDepth of quad-tree partitioning (also referred to as quad-tree partitioning), a maximum depth MaxBTDepth / a minimum depth MinBTDepth of binary-tree partitioning, a maximum transform skip block size MaxTSSize (also referred to as maximum transform skip block size), an on / off flag (also referred to as a valid flag) of each encoding tool, etc.

[0237] For example, the on / off flags for the encoding tools included in the header information Hinfo include on / off flags related to the following conversion and quantization processes. Note that the on / off flags for the encoding tools can also be interpreted as flags indicating whether or not syntax related to the encoding tool is present in the encoded data. Furthermore, if the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable, and if the value of the on / off flag is 0 (false), it indicates that the encoding tool is unusable. Note that the interpretation of the flag values ​​may be reversed.

[0238] Inter-component prediction enabled flag (ccp_enabled_flag): This flag indicates whether or not inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available. For example, if this flag information is "1" (true), it indicates that it is available, and if it is "0" (false), it indicates that it is not available. Note that this CCP is also referred to as inter-component linear prediction (CCLM or CCLMP).

[0239] Furthermore, the header information Hinfo (SPS) may include the above-mentioned operation range extension information.

[0240] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and the like.

[0241] The intra prediction mode information IPinfo includes, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from the syntax.

[0242] In addition, the intra-prediction mode information IPinfo includes, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)), a multi-class linear prediction mode flag (mclm_flag), a chrominance sample position type identifier (chroma_sample_loc_type_idx), a chrominance MPM identifier (chroma_mpm_idx), and a luminance intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0243] The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether or not inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0244] The multi-class linear prediction mode flag (mclm_flag) is information related to the linear prediction mode (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single-class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0245] The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier that identifies the type of pixel location of a chroma component (also referred to as a chroma sample location type). For example, if the chroma array type (ChromaArrayType), which is information about the color format, indicates the 420 format, the chroma sample location type identifier is assigned according to the following formula:

[0246] chroma_sample_loc_type_idx == 0:Type2 chroma_sample_loc_type_idx == 1:Type3 chroma_sample_loc_type_idx == 2:Type0 chroma_sample_loc_type_idx == 3:Type1

[0247] Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted (stored in) information about the pixel location of the chrominance component (chroma_sample_loc_info()).

[0248] The chrominance MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chrominance intra prediction mode.

[0249] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0250] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0251] <Conversion information Tinfo> The conversion information Tinfo includes, for example, the following information: Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0252] The width (TBWSize) and height (TBHSize) of the conversion block to be processed Transform skip flag (transform_skip_flag (also called ts_flag)) Scan Identifier (scanIdx) Quantization parameter (qp) Quantization matrix (scaling_matrix (e.g., JCTVC-W1005, 7.3.4 Scaling list data syntax))

[0253] Note that instead of TBWSize and TBHSize, log2TBWSize and log2TBHSize may be included in the transform information Tinfo. Log2TBWSize is the logarithmic value of TBWSize with base 2. Log2TBHSize is the logarithmic value of TBHSize with base 2. In the image decoding device 400, the transform skip flag is a flag indicating whether or not to skip the inverse coefficient transform (inverse primary transform and inverse secondary transform).

[0254] <Filter information Finfo> The filter information Finfo includes, for example, control information related to each of the following filter processes:

[0255] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0256] More specifically, for example, the information includes information specifying the picture to which each filter is applied, an area within the picture, filter On / Off control information for each CU, filter On / Off control information related to slice and tile boundaries, etc. Of course, any information may be included in the filter information Finfo, and information other than these may be included.

[0257] Returning to the description of the decoding unit 412, the decoding unit 412 derives the quantization coefficient level by referring to the syntax related to the quantization coefficient level obtained by decoding the bitstream. The decoding unit 412 supplies the quantization coefficient level to the inverse quantization unit 413.

[0258] Furthermore, the decoding unit 412 supplies encoding parameters such as the parsed header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, and filter information Finfo to each block. For example, the decoding unit 412 supplies the header information Hinfo to the inverse quantization unit 413, the inverse orthogonal transform unit 414, the prediction unit 419, and the in-loop filter unit 416. Furthermore, the decoding unit 412 supplies the prediction mode information Pinfo to the inverse quantization unit 413 and the prediction unit 419. Furthermore, the decoding unit 412 supplies the transformation information Tinfo to the inverse quantization unit 413 and the inverse orthogonal transform unit 414. Furthermore, the decoding unit 412 supplies the filter information Finfo to the in-loop filter unit 416.

[0259] Of course, the above example is merely an example and is not limiting. For example, each encoding parameter may be supplied to any processing unit. Furthermore, other information may be supplied to any processing unit.

[0260] <Inverse quantization part> The inverse quantization unit 413 has at least the components necessary to perform processing related to inverse quantization. For example, the inverse quantization unit 413 receives the transformation information Tinfo and the quantization coefficient level supplied from the decoding unit 412 as input, scales (inverse quantizes) the value of the quantization coefficient level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization. The inverse quantization unit 413 supplies the derived transformation coefficient Coeff_IQ to the inverse orthogonal transform unit 414.

[0261] Note that this inverse quantization is performed as the inverse process of the quantization performed by the quantization unit 314 of the image encoding device 300. Furthermore, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 317 of the image encoding device 300. In other words, the inverse quantization unit 317 of the image encoding device 300 performs the same process (inverse quantization) as the inverse quantization unit 413.

[0262] <Inverse orthogonal transform section> The inverse orthogonal transform unit 414 performs processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 414 receives the transform coefficients Coeff_IQ supplied from the inverse quantization unit 413 and the transform information Tinfo supplied from the decoding unit 412 as input, and performs inverse orthogonal transform processing on the transform coefficients Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficients Coeff_IQ based on the ST identifier to generate primary transform coefficients, and then performs a primary transform on the primary transform coefficients to generate a prediction residual D'. The inverse orthogonal transform unit 414 supplies the derived prediction residual D' to the calculation unit 415.

[0263] This inverse orthogonal transform is performed as the inverse process of the orthogonal transform performed by the orthogonal transform unit 313 of the image encoding device 300. This inverse orthogonal transform is the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 318 of the image encoding device 300. That is, the inverse orthogonal transform unit 318 of the image encoding device 300 performs the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 414.

[0264] Therefore, like the inverse orthogonal transform unit 318 of the image encoding device 300, the inverse orthogonal transform unit 414 can perform any inverse coefficient transform other than inverse orthogonal transform. This inverse coefficient transform is the inverse process of the coefficient transform performed by the orthogonal transform unit 313 of the image encoding device 300. In other words, the prediction residual D' may be derived by performing any inverse coefficient transform on the transform coefficients Coeff_IQ. Therefore, the inverse orthogonal transform unit 414 can also be said to be an inverse coefficient transform unit.

[0265] <Arithmetic section> The calculation unit 415 performs processing related to the addition of information related to images. For example, the calculation unit 415 receives as input a prediction residual D' supplied from the inverse orthogonal transform unit 414 and a predicted image P supplied from the prediction unit 419. As shown in the following equation, the calculation unit 415 adds the prediction residual D' and a predicted image P (prediction signal) corresponding to the prediction residual D' to derive a locally decoded image Rlocal. The calculation unit 415 supplies the derived locally decoded image Rlocal to the in-loop filter unit 416 and the frame memory 418.

[0266] Rlocal = D' + P

[0267] <In-loop filter section> The in-loop filter unit 416 performs processing related to in-loop filtering. For example, the in-loop filter unit 416 receives as input the locally decoded image Rlocal supplied from the calculation unit 415 and filter information Finfo supplied from the decoding unit 412. Note that any information may be input to the in-loop filter unit 416, and information other than the above information may also be input.

[0268] The in-loop filter unit 416 performs appropriate filtering on the locally decoded image Rlocal based on the filter information Finfo.

[0269] For example, the in-loop filter unit 416 applies four in-loop filters, namely, a bilateral filter, a deblocking filter (DBF (DeBlocking Filter)), an adaptive offset filter (SAO (Sample Adaptive Offset)), and an adaptive loop filter (ALF (Adaptive Loop Filter)), in this order, as described in Non-Patent Document 11. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate.

[0270] The in-loop filter unit 416 performs a filter process corresponding to the filter process performed on the encoding side (for example, the in-loop filter unit 320 of the image encoding device 300). Of course, the filter process performed by the in-loop filter unit 416 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 416 may apply a Wiener filter or the like.

[0271] The in-loop filter unit 416 supplies the filtered locally decoded image Rlocal to a rearrangement buffer 417 and a frame memory 418 .

[0272] <Sorting buffer> The reordering buffer 417 receives the locally decoded image Rlocal supplied from the in-loop filter unit 416 as input and holds (stores) it. The reordering buffer 417 reconstructs a decoded image R for each picture using the locally decoded image Rlocal and holds it (stores it in the buffer). The reordering buffer 417 reorders the obtained decoded images R from the decoding order to the playback order. The reordering buffer 417 outputs the reordered decoded images R to the outside of the image decoding device 400 as video data.

[0273] <Frame memory> The frame memory 418 performs processing related to the storage of image data. For example, the frame memory 418 receives the locally decoded image Rlocal supplied from the calculation unit 415 as input, reconstructs a decoded image R for each picture, and stores the reconstructed image in a buffer within the frame memory 418.

[0274] The frame memory 418 also receives as input the in-loop filtered local decoded image Rlocal supplied from the in-loop filter unit 416, reconstructs a decoded image R for each picture, and stores the reconstructed image in a buffer within the frame memory 418. The frame memory 418 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 419 as a reference image.

[0275] The frame memory 418 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to the generation of decoded images.

[0276] <Prediction Department> The prediction unit 419 performs processing related to generation of a predicted image. For example, the prediction unit 419 receives prediction mode information Pinfo supplied from the decoding unit 412 as input, performs prediction using a prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. When deriving the predicted image P, the prediction unit 419 uses, as a reference image, a decoded image R (or a part thereof) before or after filtering stored in the frame memory 418 and specified by the prediction mode information Pinfo. The prediction unit 419 supplies the derived predicted image P to the calculation unit 415.

[0277] These processing units (accumulation buffer 411 to prediction unit 419) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-described processing. Also, each processing unit may have, for example, a CPU, ROM, RAM, etc., and may execute a program using these to realize the above-described processing. Of course, each processing unit may have both of these configurations, and may realize part of the above-described processing using a logic circuit and the other part by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-described processing using a logic circuit, other processing units may execute a program to realize the above-described processing, and still other processing units may realize the above-described processing using both a logic circuit and by executing a program.

[0278] <Decryption section> FIG. 27 is a block diagram showing an example of the main configuration of the decoding unit 412. As shown in FIG. 27, the decoding unit 412 has an operation range extension information decoding unit 451. The operation range extension information decoding unit 451 performs processing related to decoding of the coded data of the operation range extension information. For example, the operation range extension information decoding unit 451 may acquire the operation range extension information extracted from the bit stream by the decoding unit 412. The operation range extension information decoding unit 451 may also acquire a parameter (BitDepth) indicating the bit depth of the image extracted from the bit stream by the decoding unit 412. The operation range extension information decoding unit 451 may use this information to perform processing related to decoding of the coded data of the operation range extension information.

[0279] Furthermore, the operation range extension information decoding unit 451 may appropriately output operation range extension information, etc. as a processing result. This operation range extension information or the internal variables corresponding thereto are supplied to any processing unit of the image decoding device 400 as encoding parameters and used therein. For example, the operation range extension information (or the internal variables) may be supplied to the decoding unit 412 and used for decoding, etc. Furthermore, the operation range extension information (or the internal variables) may be supplied to the inverse orthogonal transform unit 414 and used for inverse orthogonal transform, etc.

[0280] In the decoding unit 412 configured as above, any of the methods (method 1-1, method 2-1, or method 3) of the present technology described above may be applied.

[0281] For example, the operation range extension information decoding unit 451 may have a configuration similar to that of the operation range extension information decoding device 200 shown in Fig. 12, and may execute the operation range extension information decoding process shown in Fig. 13. By doing so, the image decoding device 400 can obtain the effect described above in <3. Control of operation range extension flag> (the effect when method 1-1 is applied). In other words, the image decoding device 400 can prevent the decoding of an unauthorized bitstream. This allows the image decoding device 400 to decode the bitstream more safely (to prevent the occurrence of failures, etc.).

[0282] Also, for example, the operation range extension information decoding unit 451 may have the same configuration as the operation range extension information decoding device 200 shown in Fig. 12 and may execute the operation range extension information decoding process shown in Fig. 18. By doing so, the image decoding device 400 can obtain the effect described above in <4. Control of the Value of Operation Range Extension Tool Control Information> (the effect when Method 2-1 is applied). In other words, the image decoding device 400 can prevent the decoding of an unauthorized bitstream. This allows the image decoding device 400 to decode the bitstream more safely (to prevent the occurrence of failures, etc.).

[0283] Also, for example, the operation range extension information decoding unit 451 may have the same configuration as the operation range extension information decoding device 200 shown in FIG. 21 and may execute the operation range extension information decoding process shown in FIG. 22. By doing so, the image decoding device 400 can obtain the effect (effect when Method 3 is applied) described above in <5. Transmission Control of Operation Range Extension Tool Control Information>. That is, the image decoding device 400 can correctly decode the bitstream generated by the image encoding device 300. That is, the image decoding device 400 can use an encoding tool for bit-depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. Also, an increase in the amount of code can be suppressed.

[0284] <Flow of image decoding process> Next, an example of the flow of image decoding processing executed by the image decoding device 400 configured as above will be described with reference to the flowchart in FIG.

[0285] When the image decoding process starts, in step S501, the accumulation buffer 411 acquires and holds (accumulates) coded data (bitstream) supplied from outside the image decoding device 400.

[0286] In step S502, the decoding unit 412 executes a decoding process to decode the coded data (bitstream) to obtain the quantization coefficients LEVEL. Furthermore, the decoding unit 412 parses (analyzes and obtains) various coding parameters from the coded data (bitstream) through this decoding.

[0287] In step S503, the inverse quantization unit 413 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantization coefficient LEVEL obtained by the process in step S502 to obtain the transform coefficient Coeff_IQ.

[0288] In step S504, the inverse orthogonal transform unit 414 performs inverse orthogonal transform processing, which is the inverse processing of the orthogonal transform processing performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S503, to obtain a prediction residual D'.

[0289] In step S505, the prediction unit 419 performs prediction processing using a prediction method specified by the encoding side based on the information parsed in step S502, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 418.

[0290] In step S506, the calculation unit 415 adds the prediction residual D' obtained in step S504 and the predicted image P obtained in step S505 to derive a locally decoded image Rlocal.

[0291] In step S507, the in-loop filter unit 416 performs in-loop filtering on the locally decoded image Rlocal obtained by the processing in step S506.

[0292] In step S508, the reordering buffer 417 derives decoded images R using the filtered locally decoded images R obtained in step S507, and reorders the order of the decoded images R from the decoding order to the playback order. The reordered decoded images R are output as a moving image to the outside of the image decoding device 400.

[0293] In step S509, the frame memory 418 stores at least one of the locally decoded image Rlocal obtained by the process in step S506 and the locally decoded image Rlocal after the filtering process obtained by the process in step S507.

[0294] When the process of step S509 ends, the image decoding process ends.

[0295] In step S502 of such image decoding processing, any of the above-described methods (method 1-1, method 2-1, or method 3) of the present technology may be applied.

[0296] For example, in step S502, the decoding unit 412 (the operation range extension information decoding unit 451 thereof) may decode the coded data of the operation range extension information (coding parameters) by executing the operation range extension information decoding process shown in FIG. 13. In this way, the image decoding device 400 can obtain the effect (the effect when method 1-1 is applied) described above in <3. Control of operation range extension flag>. In other words, the image decoding device 400 can prevent the decoding of an unauthorized bitstream. This allows the image decoding device 400 to decode the bitstream more safely (to prevent the occurrence of failures, etc.).

[0297] Also, for example, in step S502, the decoding unit 412 (the operation range extension information decoding unit 451 thereof) may execute the operation range extension information decoding process shown in Fig. 18. By doing so, the image decoding device 400 can obtain the effect (the effect when method 2-1 is applied) described above in <4. Control of the Value of Operation Range Extension Tool Control Information>. That is, the image decoding device 400 can prevent the decoding of an unauthorized bitstream. This allows the image decoding device 400 to decode the bitstream more safely (to prevent the occurrence of failures, etc.).

[0298] Also, for example, in step S502, the decoding unit 412 (the operation range extension information decoding unit 451 thereof) may execute the operation range extension information decoding process shown in Fig. 22. By doing so, the image decoding device 400 can obtain the effect (the effect when Method 3 is applied) described above in <5. Transmission Control of Operation Range Extension Tool Control Information>. That is, the image decoding device 400 can correctly decode the bitstream generated by the image encoding device 300. That is, the image decoding device 400 can use an encoding tool for bit-depth extension while maintaining compatibility with specifications that do not extend the bit depth of images. Also, an increase in the amount of code can be suppressed.

[0299] <8. Notes> <bit depth> In the above description, BitDepth is used as information indicating the bit depth of an image. However, any information can be used to obtain the bit depth of an image, and this example is not limiting. For example, a parameter sps_bitdepth_minus8 indicating "bit depth - 8" may be used. This parameter is stored in the sequence parameter set. By applying this parameter (sps_bitdepth_minus8), the above-mentioned determination of "BitDepth<=10" can be expressed as "sps_bitdepth_minus8<=2". Furthermore, the above-mentioned determination of "BitDepth>10" can be expressed as "sps_bitdepth_minus8>2".

[0300] <Storage location> In the above description, the operation range extension information (such as the operation range extension flag and operation range extension tool control information) is stored in the sequence parameter set, but the operation range extension information may be stored in any location and is not limited to this example. The operation range extension information may be stored in, for example, a picture parameter set (PPS) or a slice header (slice_header).

[0301] <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0302] FIG. 29 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0303] In a computer 900 shown in FIG. 29, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are interconnected via a bus 904.

[0304] An input / output interface 910 is also connected to the bus 904. To the input / output interface 910, an input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected.

[0305] The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 912 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 914 includes, for example, a network interface. The drive 915 drives removable media 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0306] In a computer configured as above, the CPU 901 performs the above-described series of processes by, for example, loading a program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executing the program. The RAM 903 also stores data necessary for the CPU 901 to execute various processes as appropriate.

[0307] The program executed by the computer can be applied by recording it on removable media 921 such as package media, for example. In this case, the program can be installed in storage unit 913 via input / output interface 910 by inserting removable media 921 into drive 915.

[0308] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 914 and installed in the storage unit 913.

[0309] Alternatively, this program can be installed in advance in the ROM 902 or the storage unit 913 .

[0310] <Applicable targets of this technology> This technology can be applied to any image encoding or decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding and decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. Furthermore, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0311] The present technology can also be applied to a multi-viewpoint image coding system that codes a multi-viewpoint image including images from a plurality of views. The present technology can also be applied to a multi-viewpoint image decoding system that decodes coded data of a multi-viewpoint image including images from a plurality of views. In that case, the present technology can be applied in the coding and decoding of each view.

[0312] Furthermore, the present technology can be applied to a hierarchical image coding (scalable coding) system that codes hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. Also, the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes coded data of hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. In this case, the present technology can be applied in the coding and decoding of each layer.

[0313] Furthermore, the present technology can be applied to any configuration. For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (for example, television receivers and mobile phones) in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communication, or devices (for example, hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0314] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0315] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0316] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device with multiple modules housed in a single housing, are both systems.

[0317] <Fields and uses where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, for example, transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, nature monitoring, etc. In addition, their uses are also arbitrary.

[0318] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0319] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0320] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0321] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0322] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0323] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0324] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0325] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0326] Furthermore, the program executed by the computer may have the following features. For example, the processing of the steps of writing the program may be executed in chronological order according to the order described in this specification. The processing of the steps of writing the program may also be executed in parallel. Furthermore, the processing of the steps of writing the program may be executed individually at the necessary timing, such as when called. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of writing the program may be executed in parallel with the processing of another program. Furthermore, the processing of the steps of writing the program may be executed in combination with the processing of another program.

[0327] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0328] The present technology can also be configured as follows. (1) an operation range extension flag encoding unit that encodes an operation range extension flag according to the bit depth of an image; an operation range extension tool control information encoding unit that encodes operation range extension tool control information in accordance with the operation range extension flag; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device. (2) The operation range extension flag encoding unit If the bit depth is not to be extended, the operation range extension flag is set to false and the encoding is performed; If the bit depth is extended, the operation range extension flag is encoded. The image processing device according to (1). (3) The operation range extension flag encoding unit If the bit depth is 10 bits or less, encoding the data by setting the operation range extension flag to false; If the bit depth is greater than 10 bits, encode the operation range extension flag. (2) An image processing device according to the present invention. (4) The operation range extension tool control information encoding unit skips encoding of the operation range extension tool control information when the operation range extension flag is false. An image processing device according to (2) or (3). (5) The operation range extension tool control information encoding unit sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. (4) An image processing device according to (4). (6) The operation range extension tool control information encoding unit encodes the operation range extension tool control information when the operation range extension flag is true. An image processing device according to (4) or (5). (7) The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. An image processing device according to any one of (1) to (6). (8) The operation range extension tool control information encoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. (7) An image processing device according to (7). (9) The operation range extension tool control information includes a control flag for extending the Rice parameter of TSRC (Transform Skip Residual Coding). An image processing device according to any one of (1) to (8). (10) The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. An image processing device according to any one of (1) to (9). (11) Encoding an operation range extension flag according to the bit depth of the image; encoding operation range extension tool control information according to the operation range extension flag; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

[0329] (21) an operation range extension flag encoding unit that encodes an operation range extension flag; an operation range extension tool control information encoding unit that encodes operation range extension tool control information according to the bit depth of the image; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device. (22) The operation range extension tool control information encoding unit If the operation range extension flag is true and the bit depth is not extended, encoding the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unavailable; If the operation range extension flag is true and the bit depth is to be extended, encode the operation range extension tool control information. (21) An image processing device according to (21). (23) The operation range extension tool control information encoding unit If the operation range extension flag is true and the bit depth is 10 bits or less, encoding the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unavailable; If the operation range extension flag is true and the bit depth is greater than 10 bits, encode the operation range extension tool control information. (22) An image processing device according to (22). (24) The operation range extension tool control information encoding unit skips encoding of the operation range extension tool control information when the operation range extension flag is false. The image processing device according to (22) or (23). (25) The operation range extension tool control information encoding unit sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. (24) An image processing device according to (24). (26) The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. An image processing device according to any one of (21) to (25). (27) The operation range extension tool control information encoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. (26) An image processing device according to (26). (28) The operation range extension tool control information includes a control flag for extending a Rice parameter of TSRC (Transform Skip Residual Coding). An image processing device according to any one of (21) to (27). (29) The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. An image processing device according to any one of (21) to (28). (30) Encode the operation range extension flag; Encoding operation range extension tool control information according to the bit depth of the image; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

[0330] (41) an operation range extension flag decoding unit that decodes coded data of the operation range extension flag; an operation range extension tool control information decoder that decodes coded data of the operation range extension tool control information in accordance with the operation range extension flag; a checker that checks whether there are any constraint violations in the image bitstream; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device. (42) When the operation range extension flag is true, the confirmation unit confirms whether there is a constraint violation in the bitstream. (41) An image processing device according to (41). (43) The checking unit checks whether the bit depth of the image is extended in the bitstream. (42) An image processing device according to (42). (44) The checking unit checks whether the bit depth in the bitstream is greater than 10 bits. (43) An image processing device according to (43). (45) The operation range extension tool control information decoding unit decodes the coded data of the operation range extension tool control information when there is no constraint violation in the bit stream. An image processing device according to any one of (42) to (44). (46) The confirmation unit notifies the user that the bitstream does not satisfy the constraints if the bitstream violates the constraints. An image processing device according to any one of (42) to (45). (47) If the operation range extension flag is true, the operation range extension tool control information decoding unit decodes the coded data of the operation range extension tool control information, The checking unit checks whether there is a constraint violation in the bitstream using the operation range extension tool control information. An image processing device according to any one of (41) to (46). (48) The confirmation unit confirms whether the bit depth of the image in the bitstream has not been extended and whether the operation range extension tool control information is a value indicating that the operation range extension tool is unusable. (47) An image processing device according to (47). (49) The confirmation unit confirms whether the bit depth in the bitstream is 10 bits or less and whether the operation range extension tool control information is a value indicating that the operation range extension tool is unusable. (48) An image processing device according to (48). (50) If the bitstream violates a constraint, the verification unit notifies the user that the bitstream does not satisfy the constraint. An image processing device according to any one of (47) to (49). (51) When the operation range extension flag is false, the operation range extension tool control information decoder skips decoding of the coded data of the operation range extension tool control information and sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. An image processing device according to any one of (41) to (50). (52) The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. An image processing device according to any one of (41) to (51). (53) The operation range extension tool control information decoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. (52) An image processing device according to (52). (54) The operation range extension tool control information includes a control flag for extending a Rice parameter of TSRC (Transform Skip Residual Coding). An image processing device according to any one of (41) to (53). (55) The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. An image processing device according to any one of (41) to (54). (56) Decode the coded data of the operation range extension flag; decoding the coded data of the operation range extension tool control information in accordance with the operation range extension flag; Check the image bitstream for constraint violations, the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

[0331] (61) an operation range extension flag encoding unit that encodes an operation range extension flag; an operation range extension tool control information encoding unit that encodes operation range extension tool control information according to the operation range extension flag and the bit depth of the image; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device. (62) The operation range extension tool control information encoding unit If the operation range extension flag is false or the bit depth is not extended, skip encoding the operation range extension tool control information; If the operation range extension flag is true and the bit depth is to be extended, encode the operation range extension tool control information. (61) An image processing device according to (61). (63) The operation range extension tool control information encoding unit If the operation range extension flag is false or the bit depth is 10 bits or less, skip encoding the operation range extension tool control information; If the operation range extension flag is true and the bit depth is greater than 10 bits, encode the operation range extension tool control information. (62) An image processing device according to (62). (64) If the operation range extension flag is false or the bit depth is not extended, the operation range extension tool control information encoding unit further sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. The image processing device according to (62) or (63). (65) If the operation range extension flag is false or the bit depth is 10 bits or less, the operation range extension tool control information encoding unit further sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. (64) An image processing device according to (64). (66) The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. An image processing device according to any one of (61) to (65). (67) The operation range extension tool control information encoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. (66) An image processing device according to (66). (68) The operation range extension tool control information includes a control flag for extending a Rice parameter of TSRC (Transform Skip Residual Coding). An image processing device according to any one of (61) to (67). (69) The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. An image processing device according to any one of (61) to (68). (70) Encodes the operation range extension flag, encoding operation range extension tool control information according to the operation range extension flag and the bit depth of the image; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

[0332] (81) an operation range extension flag decoding unit that decodes coded data of the operation range extension flag; an operation range extension tool control information decoding unit that decodes coded data of the operation range extension tool control information according to the operation range extension flag and the bit depth of the image; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device. (82) The operation range extension tool control information decoding unit If the operation range extension flag is false or if the bit depth is not extended, skip decoding the coded data of the operation range extension tool control information, and set the operation range extension tool control information to a value indicating that the operation range extension tool is unavailable; If the operation range extension flag is true and the bit depth is to be extended, decoding the coded data of the operation range extension tool control information. (81) An image processing device according to (81). (83) The operation range extension tool control information decoding unit If the operation range extension flag is false or the bit depth is 10 bits or less, skip decoding the coded data of the operation range extension tool control information, and set the operation range extension tool control information to a value indicating that the operation range extension tool is unavailable; If the operation range extension flag is true and the bit depth is greater than 10 bits, decode the coded data of the operation range extension tool control information. (82) An image processing device according to (82). (84) The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. An image processing device according to any one of (81) to (83). (85) The operation range extension tool control information decoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. (84) An image processing device according to (84). (86) The operation range extension tool control information includes a control flag for extending the Rice parameter of TSRC (Transform Skip Residual Coding). An image processing device according to any one of (81) to (85). (87) The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. An image processing device according to any one of (81) to (86). (88) Decode the coded data of the operation range extension flag; decoding the coded data of the operation range extension tool control information according to the operation range extension flag and the bit depth of the image; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods. [Explanation of symbols]

[0333] 100 operation range extension information encoding device, 111 operation range extension flag encoding unit, 112 operation range extension tool control information encoding unit, 200 operation range extension information decoding device, 211 operation range extension flag decoding unit, 212 operation range extension tool control information decoding unit, 213 constraint violation processing unit, 300 image encoding device, 315 encoding unit, 351 operation range extension information encoding unit, 400 image decoding device, 412 decoding unit, 451 operation range extension information decoding unit, 900 computer

Claims

1. an operation range extension flag encoding unit that encodes an operation range extension flag according to the bit depth of the image; an operation range extension tool control information encoding unit that encodes operation range extension tool control information in accordance with the operation range extension flag; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device.

2. The operation range extension flag encoding unit If the bit depth is not to be extended, the operation range extension flag is set to false and the encoding is performed; If the bit depth is extended, the operation range extension flag is encoded. The image processing device according to claim 1 .

3. The operation range extension flag encoding unit If the bit depth is 10 bits or less, encoding the operation range extension flag by setting it to false; If the bit depth is greater than 10 bits, encode the operation range extension flag. The image processing device according to claim 2 .

4. The operation range extension tool control information encoding unit skips encoding of the operation range extension tool control information when the operation range extension flag is false. The image processing device according to claim 2 .

5. The operation range extension tool control information encoding unit sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. The image processing device according to claim 4 .

6. The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. The image processing device according to claim 1 .

7. The operation range extension tool control information encoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. The image processing device according to claim 6 .

8. The operation range extension tool control information includes a control flag for extending the Rice parameter of TSRC (Transform Skip Residual Coding). The image processing device according to claim 1 .

9. The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. The image processing device according to claim 1 .

10. Encoding an operation range extension flag according to the bit depth of the image; encoding operation range extension tool control information according to the operation range extension flag; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

11. an operation range extension flag encoding unit that encodes an operation range extension flag; an operation range extension tool control information encoding unit that encodes operation range extension tool control information according to the bit depth of the image; Equipped with the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing device.

12. The operation range extension tool control information encoding unit If the operation range extension flag is true and the bit depth is not extended, encoding the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unavailable; If the operation range extension flag is true and the bit depth is to be extended, encode the operation range extension tool control information. The image processing device according to claim 11 .

13. The operation range extension tool control information encoding unit If the operation range extension flag is true and the bit depth is 10 bits or less, encoding the operation range extension tool control information by setting it to a value indicating that the operation range extension tool is unusable; If the operation range extension flag is true and the bit depth is greater than 10 bits, encode the operation range extension tool control information. The image processing device according to claim 12.

14. The operation range extension tool control information encoding unit skips encoding of the operation range extension tool control information when the operation range extension flag is false. The image processing device according to claim 12.

15. The operation range extension tool control information encoding unit sets the operation range extension tool control information to a value indicating that the operation range extension tool is unusable. The image processing device according to claim 14.

16. The operation range extension tool control information includes a control flag related to extension of the internal calculation precision of the orthogonal transform. The image processing device according to claim 11 .

17. The operation range extension tool control information encoding unit sets an internal variable related to extension of the internal calculation precision of the orthogonal transform to the value of the control flag. The image processing device according to claim 16.

18. The operation range extension tool control information includes a control flag for extending the Rice parameter of TSRC (Transform Skip Residual Coding). The image processing device according to claim 11 .

19. The operation range extension tool control information includes a control flag related to extension of RRC (Regular Residual Coding) Rice parameters. The image processing device according to claim 11 .

20. Encode the operation range extension flag, Encoding operation range extension tool control information according to the bit depth of the image; the operation range extension flag is flag information regarding extension of the operation range of the image, The operation range extension tool control information is control information for an operation range extension tool used when the operation range is extended. Image processing methods.

Images