Image processing device and method

Abstract

The present disclosure relates to an image processing device and method that make it possible to suppress an increase in the load of inverse adaptive color transform process, while suppressing an increase in distortion of coefficient data after the inverse adaptive color transform. Coefficient data obtained by reversible adaptive color transform is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at the level is subjected to the reversible inverse adaptive color transform. The present disclosure is applicable to, for example, an image processing device, an image coding device, an image decoding device, a transmit device, a receive device, a transmit / receive device, an information processing device, an image capture device, a playback device, an electronic apparatus, an image processing method, an information processing method and the like.

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method capable of suppressing an increase in the load of inverse adaptive color transform processing while suppressing an increase in distortion of coefficient data after inverse adaptive color transform. [Background technology]

[0002] Conventionally, a coding method has been proposed in which a prediction residual of a video image is derived, coefficient transformed, quantized, and coded (see, for example, Non-Patent Document 1 and Non-Patent Document 2). Also, as a coding tool for improving coding efficiency in RGB444, adaptive color transform (ACT) has been proposed, which transforms coefficient data in the RGB domain into coefficient data in the YCgCo domain. Furthermore, a reversible method has been proposed as the adaptive color transform (see, for example, Non-Patent Document 3).

[0003] Incidentally, in lossy inverse adaptive color transform (Inverse ACT), in order to suppress an increase in the load of the inverse adaptive color transform process, it has been proposed to clip the coefficient data of the YCgCo domain, which is the input signal, at [-2^bitDepth, 2^bitDepth-1] (see, for example, non-patent document 4). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, "Versatile Video Coding (Draft 7)", JVET-P2001-vE, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Feneva, CH, 1-11 Oct 2019

Non-Patent Document 2

Non-Patent Document 3

Non-Patent Document 4

Summary of the Invention

[0005] However, the value range of coefficient data in the YCgCo domain that has been adaptively color transformed using a lossless method is wider than the value range of coefficient data in the YCgCo domain that has been adaptively color transformed using a lossy method. Therefore, when the method described in Non-Patent Document 4 is applied to the lossless inverse adaptive color transform disclosed in Non-Patent Document 3, the coefficient data in the YCgCo domain changes due to clipping, and there is a risk that the distortion of the coefficient data in the RGB domain after the inverse adaptive color transform increases.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the load of the reverse adaptive color conversion process while suppressing an increase in distortion of coefficient data after reverse adaptive color conversion. [Means for solving the problem]

[0007] According to one aspect of the present technology, there is provided an image processing device comprising: A decoding unit that decodes encoded data and generates quantized coefficients, an inverse quantization unit that inverse quantizes the quantized coefficients generated by the decoding unit and generates orthogonal transform coefficients, and an inverse orthogonal transform unit that inversely orthogonally transforms the orthogonal transform coefficients generated by the inverse quantization unit, The coefficient data that has been adaptively transformed using a lossless method and an inverse orthogonal transform unit that generates the coefficient data by the inverse orthogonal transform unit. , the bit depth of the coefficient data The upper limit is the power of 2 with 1 added to it and 1 subtracted from it, and the lower limit is the power of 2 with 1 added to it and -1 multiplied by it. a clipping processing unit for clipping the signal; The upper limit value and the lower limit value and an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped by the lossless method. An image processing method according to one aspect of the present technology includes decoding encoded data, generating quantized coefficients, inverse quantizing the generated quantized coefficients, generating orthogonal transform coefficients, and performing inverse orthogonal transform on the generated orthogonal transform coefficients to generate coefficient data that has been adaptively color transformed in a lossless manner, clipping the generated coefficient data to a value obtained by subtracting 1 from a power of 2 where the exponent is a value obtained by adding 1 to a bit depth of the coefficient data as an upper limit value and a value obtained by multiplying a power of 2 where the exponent is a value obtained by adding 1 to the bit depth as an upper limit value and by multiplying the power of 2 where the exponent is a value obtained by adding 1 to the bit depth as an lower limit value, and performing inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value in the lossless manner.

[0008] According to another aspect of the present technology, there is provided an image processing device including: a transform quantization unit that performs adaptive color transform on image data in a lossless manner to generate coefficient data, orthogonally transforms the coefficient data to generate orthogonal transform coefficients, and quantizes the orthogonal transform coefficients to generate quantized coefficients; a coding unit that codes the quantized coefficients generated by the transform quantization unit to generate coded data; an inverse quantization unit that dequantizes the quantized coefficients generated by the transform quantization unit to generate the orthogonal transform coefficients; and an inverse orthogonal transform unit that inversely orthogonally transforms the orthogonal transform coefficients generated by the inverse quantization unit to generate coded data. The image processing device includes an inverse orthogonal transformation unit that generates the coefficient data that has been adaptively color transformed, a clipping processing unit that clips the coefficient data generated by the inverse orthogonal transformation unit to an upper limit value obtained by subtracting 1 from a power of 2 where the exponent is a value obtained by adding 1 to a bit depth of the coefficient data, and a lower limit value obtained by multiplying the power of 2 where the exponent is a value obtained by adding 1 to the bit depth and -1, and an inverse adaptive color transformation unit that inversely adaptively color transforms the coefficient data clipped at the upper limit value and the lower limit value by the clipping processing unit in the reversible manner. This technology other The side image processing method is as follows: The method comprises: performing an adaptive color transform on image data in a lossless manner to generate coefficient data; performing an orthogonal transform on the coefficient data to generate orthogonal transform coefficients; quantizing the orthogonal transform coefficients to generate quantized coefficients; encoding the generated quantized coefficients to generate encoded data; dequantizing the generated quantized coefficients to generate orthogonal transform coefficients; and performing an inverse orthogonal transform on the generated orthogonal transform coefficients to generate encoded data. Lossless adaptive color conversion The above Coefficient data the generated coefficient data is clipped to a value obtained by subtracting 1 from a power of 2, the value being an exponent obtained by adding 1 to the bit depth of the coefficient data, as an upper limit value, and a value obtained by multiplying -1 by a power of 2, the value being an exponent obtained by adding 1 to the bit depth, as a lower limit value, and the coefficient data clipped at the upper limit value and the lower limit value is The image processing method performs reversible adaptive color conversion.

[0009] In one aspect of the present technology, an image processing device and method include: The encoded data is decoded to generate quantized coefficients, the generated quantized coefficients are inverse quantized to generate orthogonal transform coefficients, and the generated orthogonal transform coefficients are inversely orthogonally transformed to generate The coefficient data that has been adaptively transformed using a lossless method The generated coefficient data is , the bit depth of the coefficient data is The upper limit is the power of 2 minus 1, with 1 added to the exponent, and the lower limit is the power of 2 plus 1 multiplied by -1.Clipped and its The upper and lower limits The clipped coefficient data is then inversely adaptively color transformed in a lossless manner. In an image processing device and method according to another aspect of the present technology, image data is adaptively color transformed in a lossless manner to generate coefficient data, the coefficient data is orthogonally transformed to generate orthogonal transform coefficients, the orthogonal transform coefficients are quantized to generate quantized coefficients, the generated quantized coefficients are encoded to generate encoded data, the generated quantized coefficients are inverse quantized to generate orthogonal transform coefficients, and the generated orthogonal transform coefficients are inverse orthogonally transformed to generate coefficient data that has been adaptively color transformed in a lossless manner, the generated coefficient data is clipped with an upper limit value obtained by subtracting 1 from a power of 2 whose exponent is a value obtained by adding 1 to the bit depth of the coefficient data and using a value obtained by multiplying the power of 2 whose exponent is a value obtained by adding 1 to the bit depth and using −1 as a lower limit value, and the coefficient data clipped at the upper and lower limits is inverse adaptively color transformed in the lossless manner. [Brief description of the drawings]

[0010] [Figure 1] 1A and 1B are diagrams illustrating an example of the difference in range between YCgCo conversion and YCgCo-R conversion. [Diagram 2] 1 is a block diagram showing an example of the main configuration of an inverse adaptive color conversion device. [Diagram 3] 13 is a flowchart showing an example of the flow of an inverse adaptive color conversion process. [Figure 4] 1 is a block diagram showing an example of the main configuration of an inverse quantization and inverse transform device. [Diagram 5] 13 is a flowchart showing an example of the flow of an inverse quantization and inverse transform process. [Figure 6] FIG. 2 is a block diagram showing an example of the main configuration of an image decoding device. [Figure 7] 13 is a flowchart showing an example of the flow of an image decoding process. [Figure 8] FIG. 1 is a block diagram showing an example of the main configuration of an image encoding device. [Figure 9] 11 is a block diagram showing an example of the main configuration of a transform and quantization unit. FIG. [Figure 10] 4 is a block diagram showing an example of the main configuration of an adaptive color conversion unit; [Figure 11] 13 is a flowchart showing an example of the flow of an image encoding process. [Figure 12] 11 is a flowchart illustrating an example of the flow of a transform and quantization process. [Figure 13] 11 is a flowchart illustrating an example of the flow of adaptive color conversion processing. [Figure 14] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Modes for carrying out the present disclosure (hereinafter, referred to as embodiments) will be described below in the following order. 1. Clipping of lossless inverse adaptive color transform 2. First embodiment (inverse adaptive color conversion device) 3. Second embodiment (inverse quantization and inverse transform device) 4. Third embodiment (image decoding device) 5. Fourth embodiment (image encoding device) 6. Notes

[0012] <1. Clipping process of lossless inverse adaptive color transformation> <References that support technical content and technical terms> The scope of the disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application, and the contents of other documents referenced in the following non-patent documents.

[0013] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above) Non-Patent Document 5: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-Patent Document 6: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0014] In other words, the contents described in the above non-patent documents are also the basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above non-patent documents are not directly described in the examples, they are within the disclosure scope of the present technology and meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are within the disclosure scope of the present technology and meet the support requirements of the claims.

[0015] In addition, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) indicates any partial region in a picture, and its size, shape, characteristics, etc. are not limited, unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), subblock, macroblock, tile, or slice described in the above-mentioned non-patent document.

[0016] In addition, when specifying the size of such a block, the block size may be specified indirectly in addition to directly. For example, the block size may be specified using identification information for identifying the size. For example, the block size may be specified by a ratio or difference with respect to the size of a reference block (for example, LCU or SCU). For example, when transmitting information specifying a block size as a syntax element or the like, the information may be used to indirectly specify the size as described above. In this way, the amount of information can be reduced, and coding efficiency may be improved. The specification of the block size also includes specification of a range of block sizes (for example, specification of a range of allowable block sizes, etc.).

[0017] In this specification, encoding includes not only the whole process of converting an image into a bit stream, but also a part of the process. For example, it includes not only a process including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also a process collectively referring to quantization and arithmetic coding, a process including prediction processing, quantization, and arithmetic coding, etc. Similarly, decoding includes not only the whole process of converting a bit stream into an image, but also a part of the process. For example, it includes not only a process including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also a process including inverse arithmetic decoding and inverse quantization, a process including inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0018] <Adaptive color conversion> As a coding tool for improving coding efficiency in RGB444, adaptive color transform (ACT) has been proposed, which converts coefficient data in the RGB domain into coefficient data in the YCgCo domain. This adaptive color transform is also called YCgCo transform. The inverse process of adaptive color transform is also called inverse adaptive color transform (Inverse ACT). The inverse process of YCgCo transform is also called inverse YCgCo transform. This YCgCo transform (inverse YCgCo transform) is a lossy method. Therefore, in YCgCo transform (inverse YCgCo transform), the conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain may be lossy (lossy). In other words, by converting coefficient data in the RGB domain into YCgCo and then converting it into inverse YCgCo, it may be difficult to generate (restore) coefficient data in the RGB domain that is the same as the coefficient data in the RGB domain before the YCgCo transform.

[0019] In response to this, Non-Patent Document 3 has proposed a lossless method for the adaptive color conversion. This lossless adaptive color conversion is also called YCgCo-R conversion. The inverse process of YCgCo-R conversion is also called inverse YCgCo-R conversion. In other words, this YCgCo-R conversion (inverse YCgCo-R conversion) is a lossless method. Therefore, in the YCgCo-R conversion (inverse YCgCo-R conversion), lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain can be realized. In other words, by converting the coefficient data in the RGB domain into YCgCo-R and then converting it into inverse YCgCo-R, it is possible to generate (restore) coefficient data in the RGB domain that is the same as the coefficient data in the RGB domain before the YCgCo-R conversion.

[0020] Therefore, in the case of encoding using adaptive color transform and inverse adaptive color transform, it may be difficult to realize lossless encoding (lossless coding) when lossy YCgCo transform and inverse YCgCo transform are applied. In contrast, lossless encoding (lossless coding) can be realized by applying YCgCo-R transform and inverse YCgCo-R transform.

[0021] <CgCo-R Conversion and Inverse YCgCo-R Conversion> The YCgCo-R conversion converts RGB components to YCgCo components as shown in the following formulas (1) to (4).

[0022] TIFF0007622652000001.tif832···(1) TIFF0007622652000002.tif1038···(2) TIFF0007622652000003.tif1032···(3) TIFF0007622652000004.tif1247···(4)

[0023] By the YCgCo-R conversion as shown in formulas (1) to (4), similar to the case of YCgCo conversion, the coefficient data in the RGB domain can be converted to the coefficient data in the YCgCo domain equivalent to the YCbCr domain with only simple shift operations and addition and subtraction. The inverse YCgCo-R conversion is performed as shown in the following formulas (5) to (8).

[0024] TIFF0007622652000005.tif1040···(5) TIFF0007622652000006.tif1031···(6) TIFF0007622652000007.tif1041···(7) TIFF0007622652000008.tif1037···(8)

[0025] <Clipping Process> Incidentally, Non-Patent Document 4 proposes clipping coefficient data in the YCgCo domain, which is the input signal, at [-2^bitDepth, 2^bitDepth-1] in order to suppress an increase in the load of the inverse adaptive color transform process in the lossy inverse adaptive color transform (Inverse ACT). Note that [A, B] indicates a value range with A as the lower limit and B as the upper limit. Also, "clipping at [A, B]" means that values ​​equal to or less than the lower limit A of the input signal are set as A, and values ​​equal to or more than the upper limit B are set as B. bitDepth indicates the bit depth of the coefficient data in the RGB domain before adaptive color transform.

[0026] For example, the range of coefficient data for bit depth bitDepth is [-2^bitDepth, 2^bitDepth-1]. When RGB domain coefficient data with such a range is converted to YCgCo, the range of coefficient data for each component in the YCgCo domain is theoretically within the range of [-2^bitDepth, 2^bitDepth-1]. However, in reality, due to the influence of some external factor, the coefficient data for each component in the YCgCo domain may take a value outside this range. In the inverse adaptive color conversion process, if the range of the input signal is expanded in this way, there is a risk that the load of the inverse adaptive color conversion process will increase. In particular, when implementing in hardware, there is a risk of increased costs.

[0027] Therefore, the coefficient data of each component in the YCgCo domain, which is the input signal for the inverse YCgCo conversion, is clipped at [-2^bitDepth, 2^bitDepth-1]. In other words, the coefficient data r Y [x][y], r Cb [x][y], r Cr [x][y] is processed according to the following equations (9) to (11).

[0028] TIFF0007622652000009.tif10132...(9) TIFF0007622652000010.tif9131...(10) TIFF0007622652000011.tif10133...(11)

[0029] r Y [x][y] indicates the coefficient data of the Y component. Cb [x][y] indicates the coefficient data of the Cg component. Cr [x][y] indicate the coefficient data of the Co component. Clip3(A,B,C) indicates a clip function that clips C with lower limit A and upper limit B. << indicates a bit shift (i.e., power of 2).

[0030] By doing so, there is no need to consider unnecessary processing levels outside this range in the inverse YCgCo conversion. Therefore, it is possible to suppress an increase in the load of the inverse adaptive color conversion process. In particular, when implementing in hardware, it is possible to suppress an increase in cost.

[0031] The coefficient data thus clipped is then subjected to a lossless inverse YCgCo-R transform as described in Non-Patent Document 3. In this case, the coefficient data r processed as shown in Equations (9) to (11) is Y [x][y], r Cb [x][y], r Cr [x][y] are processed according to the following equations (12) to (15).

[0032] TIFF0007622652000012.tif1285...(12) TIFF0007622652000013.tif1275...(13) TIFF0007622652000014.tif1288...(14) TIFF0007622652000015.tif1261...(15)

[0033] It should be noted that the formulas (12) to (15) are equivalent to the above-mentioned formulas (5) to (8).

[0034] <Clipping in lossless adaptive color transformation> However, in the case of the lossless YCgCo-R conversion described above, the value range of the coefficient data after conversion is wider than that of the YCgCo conversion. For example, when coefficient data of the bit depth bitDepth in the RGB domain is converted to YCgCo-R conversion, the value range of the coefficient data of the Y component, which is the luminance component, is [-2^bitDepth, 2^bitDepth-1]. In addition, the value range of the coefficient data of the Cg component and Co component, which are color difference components, is [-2^(bitDepth+1), 2^(bitDepth+1)-1]. In other words, the dynamic range of the coefficient data of the Y component is bitDepth+1, and the dynamic range of the coefficient data of the Cg component and Co component is bitDepth+2.

[0035] Therefore, when the method described in Non-Patent Document 4 is applied to the lossless adaptive color transformation disclosed in Non-Patent Document 3, the coefficient data in the YCgCo domain changes due to clipping, and there is a risk that the distortion of the coefficient data in the RGB domain after the inverse adaptive color transformation increases. Note that this "distortion" refers to the mismatch with the coefficient data in the RGB domain before the inverse adaptive color transformation, that is, the difference between the coefficient data in the RGB domain before and after the inverse adaptive color transformation. In other words, with such a method, there is a risk that it will be difficult to realize lossless transformation between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain.

[0036] 1 is a diagram showing an example of the relationship between input values ​​and output values ​​when a residual signal in the RGB domain is converted into YCgCo or YCgCo-R. The bit depth of the RGB domain coefficient data, which is the input for the YCgCo conversion and the YCgCo-R conversion, is assumed to be 10 bits (bitDepth=10). The value range of this RGB domain coefficient data is assumed to be [-2^bitDepth, 2^bitDepth-1].

[0037] A in Fig. 1 shows an example of the relationship between the G component (input value) and the Y component (output value). B in Fig. 1 shows an example of the relationship between the B component (input value) and the Cg component (output value). C in Fig. 1 shows an example of the relationship between the R component (input value) and the Co component (output value).

[0038] In addition, in Fig. 1, the grey circles show an example of the relationship between the input value and the output value in the case of YCgCo conversion. The white circles show an example of the relationship between the input value and the output value in the case of YCgCo-R conversion. The thick solid line shows an example of the upper clip value. The thick dotted line shows an example of the lower clip value. Here, the upper and lower limit values ​​of the clip processing described in Non-Patent Document 4 are shown.

[0039] As shown in FIG. 1B and FIG. 1C, it can be seen that the coefficient data of the Cg component and the coefficient data of the Co component after the YCgCo-R conversion may exceed the upper and lower clipping limits. In such a case, values ​​equal to or greater than the upper limit are clipped to the upper limit, and values ​​equal to or less than the lower limit are clipped to the lower limit. In other words, the values ​​of some of the coefficient data change. Therefore, there is a risk that distortion (mismatch with before the YCgCo-R conversion) will increase in the coefficient data after the inverse YCgCo-R conversion.

[0040] Since the inverse adaptive color transform becomes lossy in this way, it may be difficult to realize lossless coding in the coding using the inverse adaptive color transform. Also, the difference between the decoded image and the image before coding may increase due to the increase in distortion of the coefficient data in the RGB domain after the inverse adaptive color transform, which may reduce the quality of the decoded image.

[0041] <Clip range setting for lossless adaptive color transformation> Therefore, the coefficient data adaptively color transformed in the lossless method, which is the input of the lossless inverse adaptive color transform, is clipped at a level based on the bit depth of the coefficient data.

[0042] For example, in an information processing method, coefficient data that has been adaptively color transformed in a lossless manner is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at that level is then inversely adaptively color transformed in a lossless manner.

[0043] For example, an information processing device is provided with a clipping processing unit that clips coefficient data that has been adaptively color transformed in a reversible manner at a level based on the bit depth of the coefficient data, and an inverse adaptive color conversion unit that inversely adaptively converts the coefficient data clipped at that level by the clipping processing unit in a reversible manner.

[0044] In this way, it is possible to clip the input of the lossless inverse adaptive color transform at a level that does not increase the distortion of the coefficient data after the inverse adaptive color transform. Therefore, it is possible to suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in the distortion of the coefficient data after the inverse adaptive color transform.

[0045] In particular, by setting the upper and lower limit values ​​of the clipping process so that the range between the upper and lower limit values ​​includes the range of values ​​that the coefficient data adaptively color transformed in a lossless manner can theoretically take (i.e., so that the range is wider than the range of values ​​that the coefficient data can theoretically take), it is possible to suppress the occurrence of distortion in the coefficient data after the inverse adaptive color transformation. In other words, it is possible to suppress an increase in the load of the inverse adaptive color transformation process while realizing lossless transformation between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain.

[0046] <Method 1> In addition, during the clipping process, the luminance component and the color component (color difference component) of the coefficient data adaptively color transformed in a lossless manner may be clipped at the same level. The coefficient data adaptively color transformed in a lossless manner is composed of a Y component which is a luminance component, a Cg component which is a color component (color difference component), and a Co component which is a color component (color difference component). In other words, all of these components may be clipped at the same level. In this way, the clipping process can be performed more easily than when clipping is performed at different levels for each component, and an increase in the load can be suppressed.

[0047] <Method 1-1> For example, the luminance component and color component (chrominance component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2, where the power is the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner. Also, the luminance component and color component (chrominance component) of the coefficient data may be clipped using a lower limit value obtained by multiplying the power of 2, where the power is the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner, by -1. Furthermore, both such clipping of the upper limit value and clipping of the lower limit value may be performed.

[0048] For example, the upper limit actResMax of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (16) using a value obtained by adding 1 to the bit depth of the coefficient data. Also, the lower limit actResMin of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (17) using a value obtained by adding 1 to the bit depth of the coefficient data.

[0049] TIFF0007622652000016.tif1078...(16) TIFF0007622652000017.tif1078...(17)

[0050] That is, the upper limit value actResMax is a value obtained by subtracting 1 from a power of 2 obtained by adding 1 to the bit depth. The lower limit value actResMin is a value obtained by multiplying a power of 2 obtained by adding 1 to the bit depth by -1. Then, the upper limit value actResMax and the lower limit value actResMin are used to clip the luminance component and chrominance component (chrominance component) of the coefficient data adaptively color transformed in a lossless manner, as shown in the following equations (18) to (20).

[0051] TIFF0007622652000018.tif11115...(18) TIFF0007622652000019.tif10113...(19) TIFF0007622652000020.tif10112...(20)

[0052] The coefficient data thus clipped is then inverse YCgCo-R transformed as shown in the above equations (12) to (15) to derive coefficient data in the RGB domain.

[0053] In this way, the range of values ​​between the upper and lower limit values ​​of the clipping process can be made wider than the range of values ​​that the coefficient data adaptively color transformed in the lossless method can theoretically take. Therefore, by clipping at such upper and lower limit values, it is possible to suppress the occurrence of distortion in the coefficient data after the inverse adaptive color transformation. In other words, it is possible to suppress an increase in the load of the inverse adaptive color transformation process while realizing lossless transformation between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain.

[0054] <Method 1-2> Also, the clipping process may be performed taking into consideration the dynamic range of the buffer. For example, during clipping, the coefficient data adaptively color-converted in a lossless manner may be clipped at a level based on the bit depth and the dynamic range of the buffer that holds the coefficient data during the reverse adaptive color conversion. By clipping the coefficient data in this way taking into consideration the dynamic range (hardware limit) of the buffer, it is possible to suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, it is possible to suppress the increase in the dynamic range of the buffer and the increase in costs.

[0055] For example, the clipping level may be a value derived using the smaller of a value based on the bit depth of the coefficient data adaptively color transformed in a lossless manner and not based on the dynamic range of the buffer that holds the coefficient data, and a value not based on the bit depth and based on the dynamic range of the buffer. In this way, by setting the upper and lower limits using the smaller value (i.e., by clipping the coefficient data to a narrower value range), it is possible to satisfy the hardware limitations of the buffer and suppress an increase in the distortion of the coefficient data after the reverse adaptive color transformation while suppressing an increase in the load of the reverse adaptive color transformation process.

[0056] For example, the luminance component and color component (chrominance component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2, whichever is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Also, the luminance component and color component (chrominance component) of the coefficient data may be clipped using a lower limit value obtained by multiplying -1 by a power of 2, whichever is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Furthermore, both such upper limit clipping and lower limit clipping may be performed.

[0057] For example, the upper limit actResMax of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (21) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Also, the lower limit actResMin of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (22) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data.

[0058] TIFF0007622652000021.tif10130...(21) TIFF0007622652000022.tif9119...(22)

[0059] That is, the upper limit value actResMax is a value obtained by subtracting 1 from a power of 2, which is the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. The lower limit value actResMin is a value obtained by multiplying the power of 2, which is the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data, by -1. Then, using these upper limit values ​​actResMax and lower limit values ​​actResMin, the luminance component and the chrominance component (chrominance component) of the coefficient data adaptively color transformed in a lossless manner are clipped as shown in the above-mentioned equations (18) to (20).

[0060] The coefficient data thus clipped is then inverse YCgCo-R transformed as shown in the above equations (12) to (15) to derive coefficient data in the RGB domain.

[0061] In this way, it is possible to satisfy the hardware limitations of the buffer, and suppress an increase in the distortion of the coefficient data after the inverse adaptive color transform, while suppressing an increase in the load of the inverse adaptive color transform process.

[0062] <Method 2> As described above in <Clipping in lossless adaptive color conversion>, the range of coefficient data of the Cg and Co components, which are color components (color difference components), is wider than the range of coefficient data of the Y component, which is a luminance component.

[0063] Therefore, the luminance component and the chrominance component of the coefficient data may be clipped at each level. For example, the luminance component of the coefficient data may be clipped at a first level, and the chrominance component (chrominance component) of the coefficient data may be clipped at a second level. In this way, the coefficient data of each component can be clipped at a value range corresponding to the component, so that the increase in the distortion of the coefficient data after the reverse adaptive color transformation can be suppressed while the increase in the load of the reverse adaptive color transformation process can be further suppressed.

[0064] For example, if clipping is performed according to components with a wide range, the clipping width (between the upper and lower limits) for components with a narrower range becomes unnecessarily wide, and the load reduction of the reverse adaptive color conversion process is suppressed. As described above, by clipping each component with a width corresponding to that component, the increase in the load of the reverse adaptive color conversion process can be further suppressed.

[0065] <Method 2-1> The difference between the upper limit and the lower limit of the second level may be wider than the difference between the upper limit and the lower limit of the first level. As described above, the value range of the coefficient data of the Cg component and the Co component, which are chrominance components, is wider than the value range of the coefficient data of the Y component, which is a luminance component. Therefore, by making the difference between the upper limit and the lower limit of the clip for the coefficient data of the Cg component and the Co component, which are chrominance components, wider than the difference between the upper limit and the lower limit of the clip for the coefficient data of the Y component, which is a luminance component, it is possible to further suppress an increase in the load of the reverse adaptive color conversion process while suppressing an increase in the distortion of the coefficient data after the reverse adaptive color conversion.

[0066] For example, the luminance component of the coefficient data may be clipped with an upper limit value obtained by subtracting 1 from a power of 2, where the bit depth of the coefficient data adaptively color converted in a lossless manner is the exponent. The luminance component of the coefficient data may be clipped with a lower limit value obtained by multiplying the bit depth of the coefficient data adaptively color converted in a lossless manner by -1. Furthermore, both the upper limit value clipping and the lower limit value clipping may be performed for the luminance component. The color component (chrominance component) of the coefficient data may be clipped with an upper limit value obtained by subtracting 1 from a power of 2, where the bit depth of the coefficient data adaptively color converted in a lossless manner is the exponent. Furthermore, the color component (chrominance component) of the coefficient data may be clipped with a lower limit value obtained by multiplying the power of 2, where the bit depth of the coefficient data adaptively color converted in a lossless manner is the exponent, by -1. In addition, both the upper limit clipping and the lower limit clipping may be performed for the color components (chrominance components).Furthermore, the upper limit clipping and the lower limit clipping may be performed for both the luminance component and the color components (chrominance components) as described above.

[0067] For example, an upper limit actResMaxY of clipping performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (23) using the bit depth of the coefficient data. A lower limit actResMinY of clipping performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (24) using the bit depth of the coefficient data. An upper limit actResMaxC of clipping performed on the color components (chrominance components) of coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (25) using the bit depth of the coefficient data. A lower limit actResMinC of clipping performed on the color components (chrominance components) of coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (26) using the bit depth of the coefficient data.

[0068] TIFF0007622652000023.tif978...(23) TIFF0007622652000024.tif873...(24) TIFF0007622652000025.tif1088...(25) TIFF0007622652000026.tif1085...(26)

[0069] That is, the upper limit value actResMaxY for the luminance component is set to a value obtained by subtracting 1 from a power of 2 in which the bit depth is the exponent. Also, the lower limit value actResMinY for the luminance component is set to a value obtained by multiplying a power of 2 in which the bit depth is the exponent by -1. Then, using these upper limit values ​​actResMaxY and lower limit values ​​actResMinY, the luminance component of the coefficient data that has been adaptively color transformed in a lossless manner is clipped as shown in the following equation (27).

[0070] TIFF0007622652000027.tif10120...(27)

[0071] Moreover, the upper limit value actResMaxC for the color components (chrominance components) is set to a value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth. Moreover, the lower limit value actResMinC for the color components (chrominance components) is set to a value obtained by multiplying the power of 2, where the exponent is the value obtained by adding 1 to the bit depth, by -1. Then, using these upper limit values ​​actResMaxC and lower limit values ​​actResMinC, the color components (chrominance components) of the coefficient data adaptively color converted in a lossless manner are clipped, as shown in the following equations (28) and (29).

[0072] TIFF0007622652000028.tif10121...(28) TIFF0007622652000029.tif11123...(29)

[0073] The coefficient data thus clipped is then inverse YCgCo-R transformed as shown in the above equations (12) to (15) to derive coefficient data in the RGB domain.

[0074] By doing this, the coefficient data for each component can be clipped within a value range corresponding to that component, thereby further suppressing the increase in the load of the reverse adaptive color transformation process while suppressing the increase in distortion of the coefficient data after the reverse adaptive color transformation.

[0075] <Method 2-2> Also, the clipping process may be performed taking into consideration the dynamic range of the buffer. For example, during clipping, the coefficient data adaptively color-converted in a lossless manner may be clipped at a level based on the bit depth and the dynamic range of the buffer that holds the coefficient data during the inverse adaptive color conversion. That is, the above-mentioned first level and second level may be values ​​based on the bit depth of the coefficient data adaptively color-converted in a lossless manner and the dynamic range of the buffer that stores the coefficient data, respectively. By clipping the coefficient data in this way taking into consideration the dynamic range (hardware limit) of the buffer, it is possible to suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be held, it is possible to suppress the increase in the dynamic range of the buffer and the increase in costs.

[0076] For example, the level at which the luminance component is clipped may be a value derived using the smaller of a value based on the bit depth of the coefficient data adaptively color converted in a lossless manner and not based on the dynamic range of the buffer that holds the coefficient data, and a value based on the bit depth and not based on the dynamic range of the buffer. Also, the level at which the color component (chrominance component) is clipped may be a value derived using the smaller of a value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner and not based on the dynamic range of the buffer that holds the coefficient data, and a value based on the bit depth and not based on the dynamic range of the buffer. In this way, by setting the upper limit and the lower limit using the smaller value (i.e., by making the coefficient data have a narrower value range by clipping), it is possible to satisfy the hardware limitations of the buffer and suppress an increase in the distortion of each component of the coefficient data after the reverse adaptive color conversion, while suppressing an increase in the load of the reverse adaptive color conversion process.

[0077] For example, the luminance component of the coefficient data may be clipped with an upper limit value obtained by subtracting 1 from a power of 2, which is the smaller of the bit depth of the coefficient data adaptively color converted in a lossless manner and the dynamic range of the buffer that holds the coefficient data. Also, the luminance component of the coefficient data may be clipped with a lower limit value obtained by multiplying a power of 2, which is the smaller of the bit depth of the coefficient data adaptively color converted in a lossless manner and the dynamic range of the buffer that holds the coefficient data, by -1. Furthermore, both such upper limit clipping and lower limit clipping may be performed on the luminance component.

[0078] Also, the color component (chrominance component) of the coefficient data may be clipped with an upper limit value obtained by subtracting 1 from a power of 2, which is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Furthermore, the color component (chrominance component) of the coefficient data may be clipped with a lower limit value obtained by multiplying -1 by a power of 2, which is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted in a lossless manner and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Moreover, both the upper limit value clipping and the lower limit clipping for the color component (chrominance component) may be performed. Furthermore, the upper limit value clipping and the lower limit clipping for each of the luminance component and the color component (chrominance component) may all be performed as described above.

[0079] For example, the upper limit actResMaxY of the clipping process performed on the luminance component of the coefficient data adaptively color converted in the lossless manner is set as shown in the following formula (30) using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. The lower limit actResMinY of the clipping process performed on the luminance component of the coefficient data adaptively color converted in the lossless manner is set as shown in the following formula (31) using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. The upper limit actResMaxC of the clipping process performed on the color component (chrominance component) of the coefficient data adaptively color converted in the lossless manner is set as shown in the following formula (32) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. In addition, the lower limit actResMinC of the clipping process performed on the color components (chrominance components) of the coefficient data adaptively color transformed in a lossless manner is set as shown in the following equation (33) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data.

[0080] TIFF0007622652000030.tif10133...(30) TIFF0007622652000031.tif11133...(31) TIFF0007622652000032.tif10141...(32) TIFF0007622652000033.tif11139...(33)

[0081] That is, the upper limit value actResMaxY for the luminance component is set to a value obtained by subtracting 1 from a power of 2, which is the smaller of the bit depth and the dynamic range of the buffer that holds the coefficient data. The lower limit value actResMinY for the luminance component is set to a value obtained by multiplying the power of 2, which is the smaller of the bit depth and the dynamic range of the buffer that holds the coefficient data, by -1. The upper limit value actResMaxY and the lower limit value actResMinY are then used to clip the luminance component of the coefficient data that has been adaptively color transformed in a lossless manner, as shown in the above equation (27).

[0082] Moreover, the upper limit value actResMaxC for the color components (chrominance components) is set to a value obtained by subtracting 1 from a power of 2, which is the smaller of the value obtained by adding 1 to the bit depth or the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Furthermore, the lower limit value actResMinC for the color components (chrominance components) is set to a value obtained by multiplying the power of 2, which is the smaller of the value obtained by adding 1 to the bit depth or the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data, by -1. Then, using these upper limit values ​​actResMaxC and lower limit values ​​actResMinC, the color components (chrominance components) of the coefficient data that have been adaptively color converted in a lossless manner are clipped, as shown in the above-mentioned equations (28) and (29).

[0083] The coefficient data thus clipped is then inverse YCgCo-R transformed as shown in the above equations (12) to (15) to derive coefficient data in the RGB domain.

[0084] By doing so, it is possible to satisfy the hardware limitations of the buffer and further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of each component of the coefficient data after the inverse adaptive color transform.

[0085] <Method 3> The coefficient data may be clipped at a level based on the dynamic range of the buffer that holds the coefficient data during the inverse adaptive color transformation. By clipping the coefficient data in this way while taking into account the dynamic range (hardware limit) of the buffer, it is possible to prevent the buffer from overflowing. In other words, the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data to be held, so that it is possible to prevent an increase in the dynamic range of the buffer and therefore an increase in costs.

[0086] For example, the luminance component and color component (chrominance component) of the coefficient data may be clipped with an upper limit value obtained by subtracting 1 from the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data during the inverse adaptive color transformation. Also, the luminance component and color component (chrominance component) of the coefficient data may be clipped with a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1.

[0087] For example, the upper limit actResMax of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (34) using a value obtained by subtracting 1 from the dynamic range of the buffer. Also, the lower limit actResMin of the clipping process performed on coefficient data adaptively color converted in a lossless manner is set as shown in the following formula (35) using a value obtained by subtracting 1 from the dynamic range of the buffer.

[0088] TIFF0007622652000034.tif12110...(34) TIFF0007622652000035.tif12108...(35)

[0089] That is, the upper limit value actResMax is a value obtained by subtracting 1 from the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer. The lower limit value actResMin is a value obtained by multiplying the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1. Then, using the upper limit value actResMax and the lower limit value actResMin, the luminance component and the chrominance component (chrominance component) of the coefficient data adaptively color transformed in a lossless manner are clipped, as shown in the above equations (18) to (20).

[0090] The coefficient data thus clipped is then inverse YCgCo-R transformed as shown in the above equations (12) to (15) to derive coefficient data in the RGB domain.

[0091] By doing so, the hardware limitations of the buffer can be satisfied, and the occurrence of buffer overflow can be suppressed. Also, an increase in costs can be suppressed. Furthermore, since the value range of the coefficient data is limited by the clipping process, an increase in the load of the inverse adaptive color transform process can be suppressed. Furthermore, when the value range of the coefficient data is narrower than the dynamic range of the buffer, by doing so, an increase in distortion of the coefficient data after the inverse adaptive color transform can also be suppressed.

[0092] <Application Examples> Also, the above-mentioned "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3" may be selectively applied. For example, the optimal method may be applied according to a predetermined application condition based on some information such as input coefficient data, hardware specifications, and load status. For example, when the dynamic range of the buffer is wider than the value range of the coefficient data, "Method 1-1" or "Method 2-1" may be applied. Also, when the value range of the luminance component of the coefficient data and the value range of the color component (chrominance component) are the same, "Method 1", "Method 1-1", or "Method 1-2" may be applied.

[0093] In each method, either or both of the upper limit clipping and the lower limit clipping may be selectively applied. For example, when the lower limit clipping is equal to or lower than the lower limit of the coefficient data range, the lower limit clipping may be omitted (skip). When the upper limit clipping is equal to or higher than the upper limit of the coefficient data range, the upper limit clipping may be omitted (skip).

[0094] Of course, these are just examples, and the applicable conditions and the methods applied to each condition are not limited to these examples.

[0095] <2. First embodiment> <Inverse adaptive color conversion device> The present technology described above can be applied to any device. Fig. 2 is a block diagram showing an example of the configuration of an inverse adaptive color conversion device, which is one aspect of an image processing device to which the present technology is applied. The inverse adaptive color conversion device 100 shown in Fig. 2 is a device that performs a reversible inverse adaptive color conversion (inverse YCgCo-R conversion) on YCgCo domain coefficient data that has been adaptively color converted (YCgCo-R conversion) in a reversible manner from RGB domain coefficient data related to an image.

[0096] Note that Fig. 2 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 2. In other words, in the inverse adaptive color conversion device 100, there may be processing units that are not shown as blocks in Fig. 2, and there may be processing and data flows that are not shown as arrows or the like in Fig. 2.

[0097] As shown in FIG. 2, the inverse adaptive color transform device 100 includes a selection unit 101, a clip processing unit 102, and an inverse YCgCo-R transform unit 103.

[0098] The selection unit 101 acquires coefficient data res_x' to be input to the inverse adaptive color transformation device 100. The selection unit 101 also acquires cu_act_enabled_flag to be input to the inverse adaptive color transformation device 100. Cu_act_enabled_flag is flag information indicating whether adaptive color transformation (inverse adaptive color transformation) is applicable or not. When this cu_act_enabled_flag is true (e.g., "1"), it indicates that application of adaptive color transformation (inverse adaptive color transformation) is possible. When this cu_act_enabled_flag is false (e.g., "0"), it indicates that application of adaptive color transformation (inverse adaptive color transformation) is prohibited (i.e., cannot be applied).

[0099] The selection unit 101 selects whether or not to perform inverse adaptive color transformation on the coefficient data res_x' based on this cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (e.g., "1"), the selection unit 101 determines that the coefficient data res_x' is YCgCo domain coefficient data obtained by YCgCo-R transforming coefficient data in the RGB domain. Then, the selection unit 101 supplies the coefficient data res_x' to the clip processing unit 102 so that the inverse YCgCo-R transform is performed on the coefficient data res_x'.

[0100] For example, when cu_act_enabled_flag is false (e.g., "0"), the selection unit 101 determines that the coefficient data res_x' is coefficient data in the RGB domain. Then, the selection unit 101 outputs the coefficient data res_x' to the outside of the reverse adaptive color transform device 100 as coefficient data res_x after the reverse adaptive color transform. In other words, the coefficient data in the RGB domain is output to the outside of the reverse adaptive color transform device 100.

[0101] The clip processing unit 102 acquires the coefficient data res_x' supplied from the selection unit 101. The clip processing unit 102 also acquires variables such as log2MaxDR and BitDepth input to the inverse adaptive color transform device 100. log2MaxDR indicates the dynamic range of a buffer that holds the coefficient data res_x' during inverse adaptive color transform processing. BitDepth indicates the bit depth of the coefficient data res_x'.

[0102] The clipping unit 102 performs clipping on the coefficient data res_x' using upper and lower limit values ​​derived based on variables such as log2MaxDR and BitDepth. The clipping unit 102 supplies the coefficient data res_x' after the clipping to the inverse YCgCo-R conversion unit 103.

[0103] The inverse YCgCo-R converter 103 acquires the coefficient data res_x' that has been subjected to clipping, supplied from the clip processor 102. The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the acquired coefficient data res_x' to generate inverse YCgCo-R converted coefficient data res_x. The inverse YCgCo-R converter 103 outputs the generated coefficient data res_x to the outside of the inverse adaptive color converter 100. That is, the coefficient data in the RGB domain is output to the outside of the inverse adaptive color converter 100.

[0104] <Application of this technology to an inverse adaptive color conversion device> In such a reverse adaptive color transform device 100, the present technology described above in <1. Clipping process of lossless reverse adaptive color transform> can be applied.

[0105] For example, the clipping unit 102 clips the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R converter 103 performs inverse adaptive color conversion in a lossless manner on the coefficient data res_x' clipped at that level by the clipping unit 102.

[0106] In this way, the clipping unit 102 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0107] In addition, the inverse adaptive color transformation device 100 may apply the various methods of the present technology described above in <1. Clip processing of reversible inverse adaptive color transformation> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3").

[0108] For example, the clipping unit 102 may clip the luminance component and the color component (color difference component) of the coefficient data res_x' at the same level as each other, as described above in <Method 1>.

[0109] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this manner, the clipping processor 102 can perform clipping more easily and suppress an increase in the load, compared to a case where clipping is performed at a different level for each component.

[0110] For example, as described above in <Method 1-1>, the clipping unit 102 may clip the luminance component and color component (chrominance component) of the coefficient data res_x' using a value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x', as an upper limit. The clipping unit 102 may also clip the luminance component and color component (chrominance component) of the coefficient data res_x' using a value obtained by multiplying -1 by a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x', as a lower limit. Furthermore, the clipping unit 102 may perform both such clipping of the upper limit and clipping of the lower limit.

[0111] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this way, the clipping processor 102 can make the range of values ​​between the upper and lower limit values ​​of the clipping wider than the range of values ​​that the coefficient data res_x' can theoretically take. Therefore, the inverse adaptive color converter 100 can suppress the occurrence of distortion of the coefficient data res_x after the inverse adaptive color conversion. In other words, the inverse adaptive color converter 100 can suppress an increase in the load of the inverse adaptive color conversion process while realizing lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain.

[0112] Also, for example, the clipping processing unit 102 may clip the coefficient data res_x' at a level based on the bit depth and the dynamic range of the buffer that holds the coefficient data res_x' during the inverse adaptive color transformation, as described above in <Method 1-2>.

[0113] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. By clipping the coefficient data res_x' in this way while taking into consideration the dynamic range (hardware limitation) of the buffer, the inverse adaptive color converter 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without taking into consideration the bit depth of the coefficient data res_x' to be held, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color converter 100 can be suppressed.

[0114] For example, the clipping processing unit 102 may perform clipping at a level derived using the smaller of a value based on the bit depth of the coefficient data res_x' and not based on the dynamic range of the buffer holding the coefficient data res_x', and a value not based on the bit depth and based on the dynamic range of the buffer.

[0115] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this way, by setting the upper and lower limits using the smaller value (i.e., by clipping the coefficient data res_x' to a narrower value range), the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and suppress an increase in the distortion of the coefficient data res_x after the inverse adaptive color transform while suppressing an increase in the load of the inverse adaptive color transform process.

[0116] For example, the clipping unit 102 may clip the luminance component and color component (chrominance component) of the coefficient data res_x' using a value obtained by subtracting 1 from a power of 2, whichever is smaller, of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data res_x', as an upper limit value. The clipping unit 102 may also clip the luminance component and color component (chrominance component) of the coefficient data res_x' using a value obtained by multiplying -1 by a power of 2, whichever is smaller, of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data res_x', as a lower limit value. Furthermore, the clipping unit 102 may perform both such clipping of the upper limit value and clipping of the lower limit value.

[0117] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this manner, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while satisfying the hardware limitations of the buffer and suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0118] For example, the clipping unit 102 may clip the luminance component and the chrominance component of the coefficient data res_x' at their respective levels (first level and second level) as described above in <Method 2>. In this way, the clipping unit 102 can clip the coefficient data of each component at a value range corresponding to the component. The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' thus clipped, and derives the coefficient data res_x. Therefore, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0119] For example, as described above in <Method 2-1>, the difference between the upper limit and the lower limit of the second level may be wider than the difference between the upper limit and the lower limit of the first level. The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. By making the difference between the upper limit and the lower limit of the clip for the coefficient data res_x' of the Cg component and the Co component, which are color difference components, wider than the difference between the upper limit and the lower limit of the clip for the coefficient data res_x' of the Y component, which is a luminance component, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in the distortion of the coefficient data res_x after the inverse adaptive color transform.

[0120] For example, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using a value obtained by subtracting 1 from a power of 2, the bit depth of which is the exponent, as an upper limit. Also, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using a value obtained by multiplying a power of 2, the bit depth of which is the exponent, by -1 as a lower limit. Furthermore, the clipping unit 102 may perform both such upper limit clipping and lower limit clipping for the luminance component.

[0121] The clipping unit 102 may also clip the color components (chrominance components) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x'. Furthermore, the clipping unit 102 may clip the color components (chrominance components) of the coefficient data res_x' using a lower limit value obtained by multiplying a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x', by -1. Furthermore, the clipping unit 102 may perform both such clipping of the upper limit value and clipping of the lower limit value for the color components (chrominance components). Furthermore, the clipping unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value for each of the luminance component and the color components (chrominance components) as described above.

[0122] The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the luminance component and color component (color difference component) of the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this manner, the clipping processing unit 102 can clip the coefficient data res_x' of each component in a value range according to that component. Therefore, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0123] Also, for example, the clipping processing unit 102 may clip the coefficient data res_x' at a level (a first level and a second level) based on the bit depth and the dynamic range of the buffer that holds the coefficient data res_x' during the inverse adaptive color transformation, as described above in <Method 2-2>.

[0124] The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the luminance component and color component (color difference component) of the coefficient data res_x' thus clipped, and derives the coefficient data res_x. In this way, the inverse adaptive color converter 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data res_x' to be held, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color converter 100 can be suppressed.

[0125] For example, the clipping processing unit 102 may perform clipping on the luminance component of the coefficient data res_x' at a level derived using the smaller of a value based on the bit depth of the coefficient data res_x' and not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value based on the bit depth and based on the dynamic range of the buffer. Also, the clipping processing unit 102 may perform clipping on the color component (chrominance component) of the coefficient data res_x' at a level derived using the smaller of a value obtained by adding 1 to the bit depth of the coefficient data res_x' and not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value based on the bit depth and based on the dynamic range of the buffer.

[0126] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this way, by setting the upper and lower limits using the smaller value (i.e., by clipping the coefficient data to a narrower value range), the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color transform.

[0127] For example, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using a value obtained by subtracting 1 from a power of 2, which is the smaller of the bit depth of the coefficient data res_x' and the dynamic range of the buffer that holds the coefficient data res_x', as an upper limit. The clipping unit 102 may clip the luminance component of the coefficient data res_x' using a value obtained by multiplying a power of 2, which is the smaller of the bit depth of the coefficient data res_x' and the dynamic range of the buffer that holds the coefficient data res_x', by -1, as a lower limit. Furthermore, the clipping unit 102 may perform both such upper limit clipping and lower limit clipping for the luminance component.

[0128] The clipping unit 102 may also clip the color components (chrominance components) of the coefficient data res_x' using a value obtained by subtracting 1 from a power of 2, whichever is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data res_x', as an upper limit. The clipping unit 102 may also clip the color components (chrominance components) of the coefficient data res_x' using a value obtained by multiplying -1 by a power of 2, whichever is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data res_x', as a lower limit. The clipping unit 102 may also clip both the upper limit and the lower limit of the color components (chrominance components). Furthermore, the clipping processing unit 102 may clip both the upper limit value and the lower limit value for each of the luminance component and the color component (color difference component) as described above.

[0129] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, to derive the coefficient data res_x. In this manner, the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer, and can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color transform.

[0130] For example, the clipping unit 102 may clip the coefficient data res_x' at a level based on the dynamic range of a buffer that holds the coefficient data res_x' during inverse adaptive color transformation, as described above in <Method 3>.

[0131] The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, and derives the coefficient data res_x. In this way, the inverse adaptive color converter 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be stored, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color converter 100 can be suppressed.

[0132] For example, the clipping unit 102 may clip the luminance and color components (chrominance components) of the coefficient data res_x' during inverse adaptive color transformation using an upper limit value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data res_x'. The clipping unit 102 may also clip the luminance and color components (chrominance components) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1.

[0133] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, and derives the coefficient data res_x. By doing so, the inverse adaptive color converter 100 can satisfy the hardware limitations of the buffer, and can suppress the occurrence of buffer overflow. In addition, an increase in the cost of the inverse adaptive color converter 100 can be suppressed. Furthermore, since the range of the coefficient data is limited by the clipping process, the inverse adaptive color converter 100 can suppress an increase in the load of the inverse adaptive color conversion process. In addition, when the range of the coefficient data is narrower than the dynamic range of the buffer, by doing so, the inverse adaptive color converter 100 can also suppress an increase in distortion of the coefficient data after the inverse adaptive color conversion.

[0134] The reverse adaptive color transformation device 100 can be applied to the various application examples described above in <Application Examples>.

[0135] <Flow of inverse adaptive color conversion process> Next, an example of the flow of the reverse adaptive color transformation process executed by the reverse adaptive color transformation device 100 will be described with reference to the flowchart of FIG.

[0136] When the reverse adaptive color transformation process is started, the selection unit 101 of the reverse adaptive color transformation device 100 determines in step S101 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S102.

[0137] In step S102, the clipping processor 102 clips the coefficient data res_x' using predetermined upper and lower limit values.

[0138] In step S103, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' clipped in step S102, and derives coefficient data res_x after inverse adaptive color conversion.

[0139] In step S104, the inverse YCgCo-R converter 103 outputs the derived coefficient data res_x to the outside of the inverse adaptive color converter 100. When the process of step S104 ends, the inverse adaptive color conversion process ends.

[0140] Also, if it is determined in step S101 that cu_act_enabled_flag is false, the process proceeds to step S105.

[0141] In step S105, the selection unit 101 outputs the coefficient data res_x' as coefficient data res_x after the inverse adaptive color transformation to the outside of the inverse adaptive color transformation device 100. When the process of step S105 ends, the inverse adaptive color transformation process ends.

[0142] <Application of this technology to inverse adaptive color conversion processing> In such inverse adaptive color conversion processing, the present technology described above in <1. Clipping processing of lossless inverse adaptive color conversion> can be applied.

[0143] For example, in step S102, the clipping unit 102 clips the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, in step S103, the inverse YCgCo-R converter 103 performs inverse adaptive color conversion in a lossless manner on the coefficient data res_x' clipped at that level by the clipping unit 102.

[0144] In this way, the clipping unit 102 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0145] In addition, in such a reverse adaptive color conversion process, the various methods of the present technology described above in <1. Clipping process of reversible reverse adaptive color conversion> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to a reverse adaptive color conversion device>.

[0146] <3. Second embodiment> <Inverse quantization and inverse transformation device> The above-mentioned inverse adaptive color transform device 100 (inverse adaptive color transform processing) can be applied to an inverse quantization inverse transform device that executes inverse quantization inverse transform processing. FIG. 4 is a block diagram showing an example of the configuration of an inverse quantization inverse transform device, which is one aspect of an image processing device to which the present technology is applied. The inverse quantization inverse transform device 200 shown in FIG. 4 is a device in which RGB domain coefficient data related to an image is adaptively color transformed (YCgCo-R transform) in a lossless manner, orthogonally transformed, and quantized quantized coefficients are inverse quantized in a method corresponding to the quantization, inverse orthogonally transformed in a method corresponding to the orthogonal transform, and inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner. Such processing (processing executed by the inverse quantization inverse transform device 200) is also referred to as inverse quantization inverse transform processing.

[0147] Note that Fig. 4 shows the main processing units, data flows, and the like, and is not limited to all that is shown in Fig. 4. In other words, in the inverse quantization and inverse transformation device 200, there may be processing units that are not shown as blocks in Fig. 4, and there may be processing and data flows that are not shown as arrows or the like in Fig. 4.

[0148] As shown in FIG. 4, the inverse quantization and inverse transform device 200 includes an inverse quantization unit 201 , an inverse orthogonal transform unit 202 , and an inverse adaptive color transform unit 203 .

[0149] The inverse quantization unit 201 acquires a quantization coefficient qcoef_x. This quantization coefficient qcoef_x is derived by quantizing the orthogonal transform coefficient coef_x by a predetermined method. The inverse quantization unit 201 also acquires parameters necessary for inverse quantization, such as a quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag. The transform_skip_flag is flag information indicating whether or not to skip (omit) the inverse orthogonal transform process. For example, when the transform_skip_flag is true, the inverse orthogonal transform process is skipped. When the transform_skip_flag is false, the inverse orthogonal transform process is executed.

[0150] The inverse quantization unit 201 uses these parameters to inverse quantize the quantized coefficient qcoef_x by a predetermined method corresponding to the above-mentioned quantization, and derives the orthogonal transform coefficient coef_x. The inverse quantization unit 201 supplies the derived orthogonal transform coefficient coef_x to the inverse orthogonal transform unit 202.

[0151] The inverse orthogonal transform unit 202 acquires the orthogonal transform coefficient coef_x supplied from the inverse quantization unit 201. This orthogonal transform coefficient coef_x is derived by orthogonally transforming the coefficient data res_x' by a predetermined method. The inverse orthogonal transform unit 202 also acquires parameters necessary for inverse quantization, such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx. mts_idx is an identifier of MTS (Multiple Transform Selection). lfnst_idx is mode information related to low-frequency secondary transform. The inverse orthogonal transform unit 202 uses these parameters to inversely orthogonally transform the orthogonal transform coefficient coef_x by a predetermined method corresponding to the above-mentioned orthogonal transform, and derives coefficient data res_x' that has been adaptively color transformed in a lossless manner. The inverse orthogonal transform unit 202 supplies the derived coefficient data res_x' to the inverse adaptive color transform unit 203.

[0152] The inverse adaptive color transform unit 203 acquires coefficient data res_x' supplied from the inverse orthogonal transform unit 202. This coefficient data res_x' is as described in the first embodiment. The inverse adaptive color transform unit 203 also acquires cu_act_enabled_flag. The inverse adaptive color transform unit 203 appropriately performs inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner on the coefficient data res_x' based on cu_act_enabled_flag, and derives coefficient data res_x after the inverse adaptive color transform process. This coefficient data res_x is as described in the first embodiment, and is coefficient data in the RGB domain.

[0153] The inverse adaptive color transform unit 203 outputs the derived coefficient data res_x to the outside of the inverse quantization and inverse transform device 200 .

[0154] <Application of this technology to an inverse quantization and inverse transform device> The present technology described above in <1. Clipping process of inverse adaptive color transformation in a lossless system> may be applied to such an inverse quantization and inverse transformation device 200. That is, the inverse adaptive color transformation device 100 described in the first embodiment may be applied to the inverse adaptive color transformation unit 203 of the inverse quantization and inverse transformation device 200. In this case, the inverse adaptive color transformation unit 203 has the same configuration as the inverse adaptive color transformation device 100 and performs the same process.

[0155] For example, in the inverse adaptive color transform unit 203, the clipping processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clipping processing unit 102, and derives coefficient data res_x after the inverse adaptive color transform process. Then, the inverse YCgCo-R transform unit 103 outputs the derived coefficient data res_x to the outside of the inverse quantization and inverse transform device 200.

[0156] In this way, the inverse adaptive color transform unit 203 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, similar to the inverse adaptive color transform device 100, the inverse adaptive color transform unit 203 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform device 200 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the output coefficient data res_x.

[0157] Then, similarly to the case of the inverse adaptive color transformation device 100, the inverse adaptive color transformation unit 203 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transformation in a lossless system> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3"). That is, the inverse quantization and inverse transformation device 200 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transformation in a lossless system>. By applying any of these methods, the inverse quantization and inverse transformation device 200 can obtain the same effect as that described in <Application of the present technology to an inverse adaptive color transformation device>.

[0158] <Flow of inverse quantization and inverse transform processing> Next, an example of the flow of the inverse quantization and inverse transform process executed by the inverse quantization and inverse transform device 200 will be described with reference to the flowchart of FIG.

[0159] When the inverse quantization and inverse transform process starts, in step S201, the inverse quantization unit 201 of the inverse quantization and inverse transform device 200 inverse quantizes the quantized coefficients qcoef_x corresponding to each component identifier cIdx=0, 1, 2 included in the TU to be processed, and derives the orthogonal transform coefficients coef_x.

[0160] In step S202, the inverse orthogonal transform unit 202 refers to the transformation information Tinfo corresponding to each component identifier, performs inverse orthogonal transform on the orthogonal transform coefficient coef_x, and derives coefficient data res_x'.

[0161] In step S203, the inverse adaptive color transform unit 203 executes an inverse adaptive color transform process, performs an inverse YCgCo-R transform on the coefficient data res_x', and derives coefficient data res_x after the inverse adaptive color transform process.

[0162] When the process of step S203 ends, the inverse quantization and inverse transform process ends.

[0163] <Application of this technology to inverse quantization and inverse transform processing> In such an inverse quantization inverse transform process, the present technology described above in <1. Clipping process of inverse adaptive color transform of lossless method> can be applied. That is, as the inverse adaptive color transform process of step S203, the inverse adaptive color transform process described with reference to the flowchart of FIG. 3 can be applied.

[0164] In this way, the inverse adaptive color transform unit 203 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform unit 203 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in the distortion of the coefficient data res_x after the inverse adaptive color transform.

[0165] In other words, by performing the inverse quantization and inverse transform processing as described above, the inverse quantization and inverse transform device 200 can suppress an increase in the load of the inverse quantization and inverse transform processing while suppressing an increase in distortion of the output coefficient data res_x.

[0166] In this inverse quantization inverse transform process (inverse adaptive color transform process (step S203)), the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform in lossless system> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an inverse quantization inverse transform device> (i.e., the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0167] <4. Third embodiment> <Image Decoding Device> The above-mentioned inverse quantization and inverse transform device 200 (inverse quantization and inverse transform processing) can be applied to an image decoding device. FIG. 6 is a block diagram showing an example of a configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 400 shown in FIG. 6 is a device that decodes encoded data of a moving image. For example, the image decoding device 400 decodes encoded data of a moving image encoded by an encoding method such as VVC, AVC, or HEVC described in the above-mentioned non-patent document. For example, the image decoding device 400 can decode encoded data (bit stream) generated by an image encoding device 500 (FIG. 8) described later.

[0168] Note that Fig. 6 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 6. In other words, in the image decoding device 400, there may be processing units not shown as blocks in Fig. 6, and there may be processing and data flows not shown as arrows, etc. in Fig. 6. This is the same in other figures explaining processing units, etc. in the image decoding device 400.

[0169] 6, the image decoding device 400 includes a control unit 401, an accumulation buffer 411, a decoding unit 412, an inverse quantization and inverse transform unit 413, a calculation unit 414, an in-loop filter unit 415, a rearrangement buffer 416, a frame memory 417, and a prediction unit 418. Note that the prediction unit 418 includes an intra prediction unit and an inter prediction unit, not shown.

[0170] <Control Unit> The control unit 401 executes a process related to control of decoding. For example, the control unit 401 acquires coding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, etc.) included in the bitstream via the decoding unit 412. The control unit 401 may also estimate coding parameters not included in the bitstream. Furthermore, the control unit 401 controls each processing unit (accumulation buffer 411 to prediction unit 418) of the image decoding device 400 based on the acquired (or estimated) coding parameters, thereby controlling the decoding.

[0171] For example, the control unit 401 supplies header information Hinfo to the inverse quantization and inverse transform unit 413, the prediction unit 418, and the in-loop filter unit 415. In addition, the control unit 401 supplies prediction mode information Pinfo to the inverse quantization and inverse transform unit 413 and the prediction unit 418. In addition, the control unit 401 supplies transformation information Tinfo to the inverse quantization and inverse transform unit 413. In addition, the control unit 401 supplies residual information Rinfo to the decoding unit 412. In addition, the control unit 401 supplies filter information Finfo to the in-loop filter unit 415.

[0172] Of course, the above example is merely an example, and the present invention is not limited to this example. For example, each encoding parameter may be supplied to an arbitrary processing unit. In addition, other information may be supplied to an arbitrary processing unit.

[0173] <Header information Hinfo> The header information Hinfo may include information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH).

[0174] For example, the header information Hinfo may include information defining an image size (for example, parameters such as PicWidth indicating the width of an image and PicHeight indicating the height of an image). The header information Hinfo may also include information defining a bit depth (for example, parameters such as bitDepthY indicating the bit depth of a luminance component and bitDepthC indicating the bit depth of a chrominance component). The header information Hinfo may also include ChromaArrayType, which is a parameter indicating a chrominance array type. The header information Hinfo may also include parameters such as MaxCUSize indicating the maximum value of a CU size and MinCUSize indicating the minimum value of a CU size. The header information Hinfo may also include parameters such as MaxQTDepth indicating the maximum depth of quad-tree partitioning and MinQTDepth indicating the minimum depth. The header information Hinfo may also include parameters such as MaxBTDepth indicating the maximum depth of binary-tree partitioning and MinBTDepth indicating the minimum depth. Furthermore, the header information Hinfo may include a parameter such as MaxTSSize indicating the maximum value of the transform skip block (also referred to as the maximum transform skip block size).

[0175] Furthermore, the header information Hinfo may include information that specifies an on / off flag (also called a validity flag) of each encoding tool. For example, the header information Hinfo may include an on / off flag related to an orthogonal transform process or a quantization process. The on / off flag of an encoding tool may also be interpreted as a flag indicating whether or not a syntax related to that encoding tool is present in the encoded data. Furthermore, when the value of the on / off flag is 1 (true), it may indicate that the encoding tool is usable, and when the value of the on / off flag is 0 (false), it may indicate that the encoding tool is not usable. The interpretation of the flag value (true or false) may be reversed.

[0176] <Prediction mode information Pinfo> The prediction mode information Pinfo may include, for example, a parameter such as PBSize indicating the size (prediction block size) of the PB to be processed (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and other information.

[0177] The intra prediction mode information IPinfo may include, for example, information such as prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and IntraPredModeY indicating a luma intra prediction mode derived from that syntax.

[0178] Furthermore, the intra-prediction mode information IPinfo may include, for example, ccp_flag (cclmp_flag), mclm_flag, chroma_sample_loc_type_idx, chroma_mpm_idx, and information such as IntraPredModeC indicating a luma intra-prediction mode derived from these syntaxes.

[0179] ccp_flag (cclmp_flag) is an inter-component prediction flag, and is flag information indicating whether inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0180] Mclm_flag is a multi-class linear prediction mode flag, and is information on the mode of linear prediction (linear prediction mode information). More specifically, mclm_flag is flag information indicating whether or not to use the multi-class linear prediction mode. For example, when mclm_flag==0, it indicates a one-class mode (single class mode) (e.g., CCLMP), and when mclm_flag==1, it indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0181] chroma_sample_loc_type_idx is a chrominance sample location type identifier, and is an identifier that identifies the type of pixel location of the chrominance component (also called chrominance sample location type). Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted as chroma_sample_loc_info() (i.e., stored in chroma_sample_loc_info()). This chroma_sample_loc_info() is information about the pixel location of the chrominance component.

[0182] chroma_mpm_idx is a chrominance MPM identifier, and is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chrominance intra prediction mode.

[0183] The motion prediction information MVinfo may include information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0184] Of course, any information may be included in the prediction mode information Pinfo, and information other than the above information may be included in the prediction mode information Pinfo.

[0185] <Conversion information Tinfo> The conversion information Tinfo may include information such as TBWSize, TBHSize, ts_flag, scanIdx, a quantization parameter qP, and a quantization matrix scaling_matrix (see, for example, JCTVC-W1005, 7.3.4 Scaling list data syntax).

[0186] TBWSize is a parameter indicating the width size of the transform block to be processed. Note that the transform information Tinfo may include the logarithm of TBWSize with base 2 as log2TBWSize instead of this TBWSize. TBHSize is a parameter indicating the height size of the transform block to be processed. Note that the transform information Tinfo may include the logarithm of TBHSize with base 2 as log2TBHSize instead of this TBHSize.

[0187] ts_flag is a transform skip flag, which is flag information indicating whether or not to skip the (inverse) primary transform and the (inverse) secondary transform. scanIdx is a scan identifier.

[0188] Of course, any information may be included in the conversion information Tinfo, and information other than the above information may be included in the conversion information Tinfo.

[0189] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax in JCTVC-W1005) may include, for example, the following information:

[0190] cbf(coded_block_flag): Residual data presence flag last_sig_coeff_x_pos: last non-zero coefficient X coordinate last_sig_coeff_y_pos: Y coordinate of the last non-zero coefficient coded_sub_block_flag: Sub-block non-zero coefficient presence / absence flag sig_coeff_flag: Non-zero coefficient presence / absence flag gr1_flag: Flag indicating whether the level of the non-zero coefficient is greater than 1 (also called the GR1 flag) gr2_flag: Flag indicating whether the level of the non-zero coefficient is greater than 2 (also called the GR2 flag) sign_flag: The sign indicating whether a non-zero coefficient is positive or negative (also called the sign sign) coeff_abs_level_remaining: The remaining level of non-zero coefficients (also called the non-zero coefficient residual level)

[0191] Of course, the information included in the residual information Rinfo is arbitrary, and information other than the above information may be included.

[0192] <Filter information Finfo> The filter information Finfo may include, for example, control information regarding each of the filter processes shown below.

[0193] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0194] Also, for example, the information may include information specifying a picture to which each filter is applied, an area within a picture, on / off control information for a filter in units of a CU, on / off control information for a filter relating to a boundary between slices and tiles, etc. Of course, any information may be included in the filter information Finfo, and information other than these pieces of information may be included.

[0195] <Accumulation buffer> The accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400. At a predetermined timing, or when a predetermined condition is met, the accumulation buffer 411 extracts coded data included in the accumulated bitstream and supplies the coded data to the decoding unit 412.

[0196] <Decryption section> The decoding unit 412 obtains the encoded data supplied from the accumulation buffer 411, and entropy-decodes (losslessly decodes) the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, to derive encoding parameters.

[0197] The coding parameters may include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, etc. That is, the decoding unit 412 decodes and parses (analyzes and obtains) this information from the bitstream.

[0198] The decoding unit 412 executes such processing (decoding, parsing, etc.) under the control of the control unit 401, and supplies the obtained information to the control unit 401.

[0199] Furthermore, the decoding unit 412 decodes the encoded data by referring to the residual information Rinfo. At this time, the decoding unit 412 applies entropy decoding (lossless decoding) such as CABAC or CAVLC. That is, the decoding unit 412 decodes the encoded data by a decoding method corresponding to the encoding method of the encoding process executed by the encoding unit 514 of the image encoding device 500.

[0200] For example, assume that CABAC is applied. The decoding unit 412 performs arithmetic decoding using a context model on the encoded data to derive the quantization coefficient level of each coefficient position in each transform block. The decoding unit 412 supplies the derived quantization coefficient level to the inverse quantization and inverse transform unit 413.

[0201] <Inverse quantization and inverse transform section> The inverse quantization and inverse transform unit 413 obtains the quantization coefficient level supplied from the decoding unit 412. The inverse quantization and inverse transform unit 413 obtains coding parameters such as prediction mode information Pinfo and transformation information Tinfo supplied from the control unit 401.

[0202] The inverse quantization inverse transform unit 413 performs an inverse quantization inverse transform process on the quantization coefficient level based on coding parameters such as prediction mode information Pinfo and transform information Tinfo, and derives residual data D'. This inverse quantization inverse transform process is an inverse process of a transform quantization process performed in a transform quantization unit 513 (FIG. 8) described later. That is, in the inverse quantization inverse transform process, for example, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are performed. The inverse quantization is an inverse process of the quantization performed in the transform quantization unit 513. The inverse orthogonal transform is an inverse process of the orthogonal transform performed in the transform quantization unit 513. The inverse adaptive color transform is an inverse process of the adaptive color transform performed in the transform quantization unit 513. Of course, the processes included in the inverse quantization inverse transform process are arbitrary, and some of the processes described above may be omitted, or processes other than those described above may be included. The inverse quantization inverse transform unit 413 supplies the derived residual data D' to the calculation unit 414.

[0203] <Arithmetic section> The calculation unit 414 obtains the residual data D' supplied from the inverse quantization and inverse transform unit 413 and the predicted image supplied from the prediction unit 418. The calculation unit 414 adds the residual data D' and the predicted image P corresponding to the residual data D' to derive a locally decoded image. The calculation unit 414 supplies the derived locally decoded image to the in-loop filter unit 415 and the frame memory 417.

[0204] <In-loop filter section> The in-loop filter unit 415 obtains the locally decoded image supplied from the calculation unit 414. The in-loop filter unit 415 obtains filter information Finfo supplied from the control unit 401. Note that any information is input to the in-loop filter unit 415, and information other than the above information may be input.

[0205] The in-loop filter unit 415 appropriately performs a filter process on the locally decoded image based on the filter information Finfo. For example, the in-loop filter unit 415 may apply a bilateral filter as the filter process. For example, the in-loop filter unit 415 may apply a deblocking filter (DBF (DeBlocking Filter)) as the filter process. For example, the in-loop filter unit 415 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as the filter process. For example, the in-loop filter unit 415 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as the filter process. In addition, the in-loop filter unit 415 may apply a combination of a plurality of filters among these as the filter process. Note that which filter is applied and in what order it is applied are arbitrary and can be appropriately selected. For example, the in-loop filter unit 415 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filter process.

[0206] The in-loop filter unit 415 performs a filter process corresponding to the filter process performed by the encoding device. For example, the in-loop filter unit 415 performs a filter process corresponding to the filter process performed by an in-loop filter unit 518 (FIG. 8) of the image encoding device 500 described later. Of course, the filter process performed by the in-loop filter unit 415 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 415 may apply a Wiener filter or the like.

[0207] The in-loop filter unit 415 supplies the filtered locally decoded image to a reordering buffer 416 and a frame memory 417 .

[0208] <Sorting Buffer> The reordering buffer 416 receives the locally decoded image supplied from the in-loop filter unit 415 as input and holds (stores) it. The reordering buffer 416 uses the locally decoded image to reconstruct a decoded image for each picture and holds it (stores it in the buffer). The reordering buffer 416 reorders the obtained decoded images from the decoding order to the playback order. The reordering buffer 416 outputs the group of decoded images reordered in the playback order to the outside of the image decoding device 400 as video data.

[0209] <Frame memory> The frame memory 417 obtains the locally decoded image supplied from the calculation unit 414, reconstructs the decoded image for each picture unit, and stores the reconstructed image in a buffer within the frame memory 417. The frame memory 417 also obtains the locally decoded image that has been in-loop filtered and supplied from the in-loop filter unit 415, reconstructs the decoded image for each picture unit, and stores the reconstructed image in a buffer within the frame memory 417.

[0210] The frame memory 417 appropriately supplies the stored decoded image (or a part thereof) as a reference image to the prediction unit 418. Note that the frame memory 417 may also store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to generation of the decoded image.

[0211] <Prediction Department> The prediction unit 418 obtains prediction mode information Pinfo supplied from the control unit 401. The prediction unit 418 also obtains a decoded image (or a part thereof) to be read from the frame memory 417. The prediction unit 418 executes a prediction process in the prediction mode adopted at the time of encoding based on the prediction mode information Pinfo, and generates a predicted image P by referring to the decoded image as a reference image. The prediction unit 418 supplies the generated predicted image P to the calculation unit 414.

[0212] <Application of this technology to image decoding devices> The present technology described above in <1. Clipping process of lossless inverse adaptive color transform> may be applied to such an image decoding device 400. That is, the image decoding device 400 may apply the inverse quantization and inverse transform device 200 described in the second embodiment as the inverse quantization and inverse transform unit 413. In this case, the inverse quantization and inverse transform unit 413 has a configuration similar to that of the inverse quantization and inverse transform device 200 (FIG. 4) and performs similar processing.

[0213] For example, in the inverse quantization and inverse transform unit 413, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the decoding unit 412 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 inverse quantizes the quantization coefficient qcoef_x using information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag, to derive the orthogonal transform coefficient coef_x.

[0214] The inverse orthogonal transform unit 202 performs inverse orthogonal transform on the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201 using information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx, to derive coefficient data res_x'.

[0215] The inverse adaptive color transform unit 203 performs inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner as appropriate on the coefficient data res_x' derived by the inverse orthogonal transform unit 202 based on cu_act_enabled_flag, and derives coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.

[0216] As described in the first embodiment, the inverse adaptive color transform unit 203 has the same configuration as the inverse adaptive color transform device 100 and performs the same processing. For example, in the inverse adaptive color transform unit 203, the clip processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clip processing unit 102, and derives coefficient data res_x after the inverse adaptive color transform processing. Then, the inverse YCgCo-R transform unit 103 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.

[0217] In this way, the inverse quantization and inverse transform unit 413 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, similar to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 413 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in the distortion of the output coefficient data res_x (residual data D'). In other words, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0218] Then, similarly to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 413 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform in a lossless system> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3"). That is, the image decoding device 400 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform in a lossless system>. By applying any of these methods, the image decoding device 400 can obtain the same effect as that described in <Application of the present technology to an inverse quantization and inverse transform device> (that is, the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0219] <Flow of image decoding process> Next, a description will be given of the flow of each process executed by the above-described image decoding device 400. First, an example of the flow of an image decoding process will be described with reference to the flowchart in FIG.

[0220] When the image decoding process starts, in step S401, the accumulation buffer 411 acquires and holds (accumulates) a bit stream (encoded data) supplied from outside the image decoding device 400.

[0221] In step S402, the decoding unit 412 executes a decoding process. For example, the decoding unit 412 parses (analyzes and acquires) various coding parameters (for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, etc.) from the bit stream. The control unit 401 supplies the acquired various coding parameters to various processing units, thereby setting the various coding parameters.

[0222] Furthermore, the control unit 401 sets the unit of processing based on the obtained coding parameters. Furthermore, the decoding unit 412 decodes the bit stream under the control of the control unit 401, and derives the quantization coefficient level.

[0223] In step S403, the inverse quantization and inverse transform unit 413 executes an inverse quantization and inverse transform process to derive residual data D'.

[0224] In step S404, the prediction unit 418 generates a predicted image P. For example, the prediction unit 418 performs a prediction process using a prediction method specified by the encoding side based on the encoding parameters set in step S402, and generates the predicted image P by referring to a reference image stored in the frame memory 417, for example.

[0225] In step S405, the calculation unit 414 adds the residual data D' obtained in step S403 and the predicted image P obtained in step S404 to derive a locally decoded image.

[0226] In step S406, the in-loop filter unit 415 performs in-loop filtering on the locally decoded image obtained by the processing in step S405.

[0227] In step S407, the reordering buffer 416 derives decoded images using the locally decoded images filtered in step S406, and reorders the order of the decoded images from the decoding order to the playback order. The decoded images reordered in the playback order are output to the outside of the image decoding device 400 as moving images.

[0228] Furthermore, in step S408, the frame memory 417 stores at least one of the locally decoded image obtained in the process of step S405 and the locally decoded image that has been filtered in the process of step S406.

[0229] When the process of step S408 ends, the image decoding process ends.

[0230] <Application of this technology to image decoding processing> In such an image decoding process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. That is, as the inverse quantization and inverse transform process of step S403, the inverse quantization and inverse transform process described with reference to the flowchart in FIG. 5 can be applied.

[0231] In this way, the inverse quantization and inverse transform unit 413 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform unit 413 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform.

[0232] That is to say, by performing the image decoding process as described above, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0233] In this image decoding process (the inverse quantization and inverse transform process (step S403)), the various methods of the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the Present Technology to an Image Decoding Device> (i.e., the same effect as that described in <Application of the Present Technology to an Inverse Adaptive Color Transform Device>).

[0234] <5. Fourth embodiment> <Image encoding device> The above-mentioned inverse quantization and inverse transform device 200 (inverse quantization and inverse transform processing) can be applied to an image encoding device. FIG. 8 is a block diagram showing an example of a configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 500 shown in FIG. 8 is a device that encodes image data of a moving image. For example, the image encoding device 500 encodes image data of a moving image by an encoding method such as VVC (Versatile Video Coding), AVC (Advanced Video Coding), or HEVC (High Efficiency Video Coding) described in the above-mentioned non-patent document. For example, the image encoding device 500 may generate encoded data (bit stream) that can be decoded by the above-mentioned image decoding device 400 (FIG. 6).

[0235] Note that Fig. 8 shows the main processing units, data flows, etc., and is not limited to all that is shown in Fig. 8. In other words, in the image encoding device 500, there may be processing units that are not shown as blocks in Fig. 8, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 8. This is the same in other figures that explain processing units, etc. in the image encoding device 500.

[0236] 8, the image coding device 500 includes a control unit 501, a rearrangement buffer 511, a calculation unit 512, a transform / quantization unit 513, an encoding unit 514, and an accumulation buffer 515. The image coding device 500 also includes a dequantization / inverse transform unit 516, a calculation unit 517, an in-loop filter unit 518, a frame memory 519, a prediction unit 520, and a rate control unit 521.

[0237] <Control Unit> The control unit 501 divides the video data held by the rearrangement buffer 511 into blocks (CU, PU, ​​TU, etc.) of processing units based on an external or pre-specified block size of the processing units. The control unit 501 also determines the coding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization). For example, the control unit 501 can set a transformation skip flag, etc.

[0238] When the control unit 501 determines the above coding parameters, it supplies them to each block. For example, the header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the coding unit 514 and the prediction unit 520. The transformation information Tinfo is supplied to the coding unit 514, the transformation quantization unit 513, and the inverse quantization inverse transformation unit 516. The filter information Finfo is supplied to the coding unit 514 and the in-loop filter unit 518. Of course, the destination of each coding parameter is arbitrary and is not limited to this example.

[0239] <Sorting Buffer> Each field (input image) of video data is input to the image coding device 500 in its playback order (display order). The reordering buffer 511 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 501, the reordering buffer 511 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 511 supplies each processed input image to the calculation unit 512.

[0240] <Arithmetic section> The calculation unit 512 subtracts the predicted image P supplied from the prediction unit 520 from the image corresponding to the block of processing units supplied from the sorting buffer 511 to derive residual data D, and supplies it to the transformation and quantization unit 513.

[0241] <Transformation and quantization section> The transform quantization unit 513 acquires the residual data D supplied from the calculation unit 512. The transform quantization unit 513 also acquires the prediction mode information Pinfo and the transformation information Tinfo supplied from the control unit 501. The transform quantization unit 513 performs a transform quantization process on the residual data D based on the prediction mode information Pinfo and the transformation information Tinfo, and derives a quantization coefficient level. In the transform quantization process, for example, processes such as adaptive color transformation, orthogonal transformation, and quantization are performed. Of course, the processes included in the transform quantization process are arbitrary, and some of the above-mentioned processes may be omitted, or processes other than those described above may be included. The transform quantization unit 513 supplies the derived quantization coefficient level to the encoding unit 514 and the inverse quantization inverse transformation unit 516.

[0242] <Encoding part> The encoding unit 514 acquires the quantization coefficient level supplied from the transform and quantization unit 513. The encoding unit 514 also acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 501. The encoding unit 514 also acquires information related to filters, such as filter coefficients, supplied from the in-loop filter unit 518. The encoding unit 514 also acquires information related to an optimal prediction mode supplied from the prediction unit 520.

[0243] The encoding unit 514 performs entropy encoding (lossless encoding) on ​​the quantization coefficient level to generate a bit string (encoded data). The encoding unit 514 may apply, for example, CABAC (Context-based Adaptive Binary Arithmetic Code) as this entropy encoding. The encoding unit 514 may apply, for example, CAVLC (Context-based Adaptive Variable Length Code) as this entropy encoding. Of course, the content of this entropy encoding is arbitrary and is not limited to these examples.

[0244] Furthermore, the encoding unit 514 derives residual information Rinfo from the quantized coefficient levels, encodes the residual information Rinfo, and generates a bit string.

[0245] Furthermore, the encoding unit 514 includes information on the filter supplied from the in-loop filter unit 518 in the filter information Finfo, and includes information on the optimal prediction mode supplied from the prediction unit 520 in the prediction mode information Pinfo. Then, the encoding unit 514 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0246] The encoding unit 514 multiplexes the bit strings of the various pieces of information generated as described above to generate encoded data. The encoding unit 514 supplies the encoded data to the accumulation buffer 515.

[0247] <Accumulation buffer> The accumulation buffer 515 temporarily holds the encoded data obtained in the encoding unit 514. The accumulation buffer 515 outputs the held encoded data, for example as a bit stream, to the outside of the image encoding device 500 at a predetermined timing. For example, this encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, etc. In other words, the accumulation buffer 515 is also a transmission unit that transmits the encoded data (bit stream).

[0248] <Inverse quantization and inverse transform section> The inverse quantization and inverse transform unit 516 obtains the quantization coefficient level supplied from the transform and quantization unit 513. In addition, the inverse quantization and inverse transform unit 516 obtains the transformation information Tinfo supplied from the control unit 501.

[0249] The inverse quantization and inverse transform unit 516 performs an inverse quantization and inverse transform process on the quantization coefficient level based on the transform information Tinfo, and derives residual data D'. This inverse quantization and inverse transform process is an inverse process of the transform and quantization process performed in the transform and quantization unit 513, and is the same process as the inverse quantization and inverse transform process performed in the inverse quantization and inverse transform unit 413 of the image decoding device 400 described above.

[0250] That is, in the inverse quantization and inverse transform process, for example, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are executed. This inverse quantization is an inverse process of the quantization executed in the transform quantization unit 513, and is the same process as the inverse quantization executed in the inverse quantization and inverse transform unit 413. Moreover, the inverse orthogonal transform is an inverse process of the orthogonal transform executed in the transform quantization unit 513, and is the same process as the inverse orthogonal transform executed in the transform quantization unit 513. Furthermore, the inverse adaptive color transform is an inverse process of the adaptive color transform executed in the transform quantization unit 513, and is the same process as the inverse adaptive color transform executed in the transform quantization unit 513.

[0251] Of course, any process may be included in the inverse quantization and inverse transform process, and some of the processes described above may be omitted, or processes other than those described above may be included. The inverse quantization and inverse transform unit 516 supplies the derived residual data D′ to the calculation unit 517.

[0252] <Arithmetic section> The calculation unit 517 obtains the residual data D' supplied from the inverse quantization and inverse transform unit 516 and the predicted image P supplied from the prediction unit 520. The calculation unit 517 adds the residual data D' to the predicted image P corresponding to the residual data D' to derive a locally decoded image. The calculation unit 517 supplies the derived locally decoded image to the in-loop filter unit 518 and the frame memory 519.

[0253] <In-loop filter section> The in-loop filter unit 518 acquires a locally decoded image supplied from the calculation unit 517. The in-loop filter unit 518 also acquires filter information Finfo supplied from the control unit 501. The in-loop filter unit 518 also acquires an input image (original image) supplied from the rearrangement buffer 511. Any information may be input to the in-loop filter unit 518, and information other than the above information may be input. For example, a prediction mode, motion information, a code amount target value, a quantization parameter qP, a picture type, information on a block (CU, CTU, etc.), etc. may be input to the in-loop filter unit 518 as necessary.

[0254] The in-loop filter unit 518 appropriately performs a filter process on the locally decoded image based on the filter information Finfo. The in-loop filter unit 518 also uses the input image (original image) and other input information for the filter process as necessary.

[0255] For example, the in-loop filter unit 518 may apply a bilateral filter as the filtering process. For example, the in-loop filter unit 518 may apply a deblocking filter (DBF (DeBlocking Filter)) as the filtering process. For example, the in-loop filter unit 518 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as the filtering process. For example, the in-loop filter unit 518 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as the filtering process. Furthermore, the in-loop filter unit 518 may apply a combination of a plurality of these filters as the filtering process. Note that which filter is applied and in what order are optional and can be appropriately selected. For example, the in-loop filter unit 518 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filtering process.

[0256] Of course, the filtering process performed by the in-loop filter unit 518 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 518 may apply a Wiener filter or the like.

[0257] The in-loop filter unit 518 supplies the filtered locally decoded image to the frame memory 519. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 518 supplies the information about the filter to the encoding unit 514.

[0258] <Frame memory> The frame memory 519 executes processing related to storage of image-related data. For example, the frame memory 519 acquires a locally decoded image supplied from the calculation unit 517 and a locally decoded image that has been subjected to a filter process and supplied from the in-loop filter unit 518, and holds (stores) them. The frame memory 519 also reconstructs a decoded image for each picture unit using the locally decoded image, and holds it (stores it in a buffer in the frame memory 519). The frame memory 519 supplies the decoded image (or a part thereof) to the prediction unit 520 in response to a request from the prediction unit 520.

[0259] <Prediction Department> The prediction unit 520 executes a process related to generation of a predicted image. For example, the prediction unit 520 acquires prediction mode information Pinfo supplied from the control unit 501. For example, the prediction unit 520 acquires an input image (original image) supplied from the sorting buffer 511. For example, the prediction unit 520 acquires a decoded image (or a part thereof) read from the frame memory 519.

[0260] The prediction unit 520 uses the prediction mode information Pinfo and an input image (original image) to perform prediction processing such as inter prediction and intra prediction. That is, the prediction unit 520 performs prediction and motion compensation by referring to the decoded image as a reference image, and generates a predicted image P. The prediction unit 520 supplies the generated predicted image P to the calculation unit 512 and the calculation unit 517. In addition, the prediction unit 520 supplies information on the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 514 as necessary.

[0261] <Rate control section> The rate control unit 521 executes a process related to rate control. For example, the rate control unit 521 controls the rate of the quantization operation of the transform and quantization unit 513 based on the code amount of the encoded data accumulated in the accumulation buffer 515 so as to prevent overflow or underflow.

[0262] <Transformation and quantization section> Fig. 9 is a block diagram showing an example of the main configuration of the transform and quantization unit 513 in Fig. 8. As shown in Fig. 9, the transform and quantization unit 513 has an adaptive color transform unit 541, an orthogonal transform unit 542, and a quantization unit 543.

[0263] The adaptive color conversion unit 541 acquires the residual data D (FIG. 8) supplied from the calculation unit 512 as coefficient data res_x. The adaptive color conversion unit 541 also acquires cu_act_enabled_flag supplied from the control unit 501. The adaptive color conversion unit 541 executes adaptive color conversion on the coefficient data res_x based on the value of cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (for example, “1”), the adaptive color conversion unit 541 performs lossless adaptive color conversion (YCgCo-R conversion) to convert the coefficient data res_x in the RGB domain into coefficient data res_x′ in the YCgCo domain. This adaptive color conversion is the inverse process of the inverse adaptive color conversion executed in the inverse quantization inverse conversion unit 413 or the inverse quantization inverse conversion unit 516. The adaptive color conversion unit 541 supplies the derived coefficient data res_x′ to the orthogonal conversion unit 542.

[0264] The orthogonal transform unit 542 acquires coefficient data res_x' supplied from the adaptive color transform unit 541. The orthogonal transform unit 542 acquires information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501. The orthogonal transform unit 542 orthogonally transforms the coefficient data res_x' using the acquired information, and derives an orthogonal transform coefficient coef_x. This orthogonal transform is an inverse process of the inverse orthogonal transform executed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The orthogonal transform unit 542 supplies the derived orthogonal transform coefficient coef_x to the quantization unit 543.

[0265] The quantization unit 543 acquires the orthogonal transform coefficient coeff_x supplied from the orthogonal transform unit 542. The quantization unit 543 also acquires information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501. The quantization unit 543 quantizes the orthogonal transform coefficient coef_x using the acquired information, and derives the quantization coefficient qcoef_x. This quantization is an inverse process of the inverse quantization performed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The quantization unit 543 supplies the derived quantization coefficient qcoef_x to the encoding unit 514 and the inverse quantization and inverse transform unit 516 (FIG. 8) as the quantization coefficient level.

[0266] <Adaptive color conversion section> Fig. 10 is a block diagram showing an example of the main configuration of the adaptive color conversion section 541 of Fig. 9. As shown in Fig. 10, the adaptive color conversion section 541 has a selection section 571 and a YCgCo-R conversion section 572.

[0267] The selection unit 571 acquires the residual data D (FIG. 8) supplied from the calculation unit 512 as coefficient data res_x. The selection unit 571 also acquires cu_act_enabled_flag supplied from the control unit 501. The selection unit 571 selects whether or not to perform adaptive color conversion on the acquired coefficient data res_x based on the value of cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (e.g., "1"), the selection unit 571 determines that adaptive color conversion can be applied, and supplies the coefficient data res_x to the YCgCo-R conversion unit 572.

[0268] For example, if cu_act_enabled_flag is false (e.g., "0"), the selection unit 571 determines that the application of adaptive color transformation (inverse adaptive color transformation) is prohibited (i.e., cannot be applied), and supplies the coefficient data res_x to the orthogonal transformation unit 542 (Figure 9) as coefficient data res_x' after the applied color transformation.

[0269] The YCgCo-R transform unit 572 acquires the coefficient data res_x supplied from the selection unit 101. The YCgCo-R transform unit 572 performs YCgCo-R transform on the acquired coefficient data res_x, and derives YCgCo-R converted coefficient data res_x'. This YCgCo-R transform is an inverse process of the inverse YCgCo-R transform executed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The YCgCo-R transform unit 572 supplies the derived coefficient data res_x' to the orthogonal transform unit 542 (FIG. 9).

[0270] <Application of this technology to image encoding devices> In such an image coding device 500 (FIG. 8), the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> may be applied. That is, the image coding device 500 may apply the inverse quantization and inverse transform device 200 described in the second embodiment as the inverse quantization and inverse transform unit 516. In this case, the inverse quantization and inverse transform unit 516 has the same configuration as the inverse quantization and inverse transform device 200 (FIG. 4) and performs the same process.

[0271] For example, in the inverse quantization and inverse transform unit 516, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the transform and quantization unit 513 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 uses information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501 to inverse quantize the quantization coefficient qcoef_x, and derives the orthogonal transform coefficient coef_x.

[0272] The inverse orthogonal transform unit 202 performs inverse orthogonal transform on the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201 using information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501, to derive coefficient data res_x'.

[0273] The inverse adaptive color transform unit 203 performs inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner as appropriate on the coefficient data res_x' derived by the inverse orthogonal transform unit 202 based on cu_act_enabled_flag supplied from the control unit 501, to derive coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.

[0274] As described in the first embodiment, the inverse adaptive color transform unit 203 has the same configuration as the inverse adaptive color transform device 100 and performs the same processing. For example, in the inverse adaptive color transform unit 203, the clip processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clip processing unit 102, and derives coefficient data res_x after the inverse adaptive color transform processing. Then, the inverse YCgCo-R transform unit 103 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.

[0275] In this way, the inverse quantization and inverse transform unit 516 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color transform. Therefore, similar to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 516 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in the distortion of the output coefficient data res_x (residual data D'). In other words, the image encoding device 500 can suppress an increase in the load of the image encoding process while suppressing a decrease in the quality of the decoded image.

[0276] Then, similarly to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 516 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform in a lossless system> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3"). That is, the image encoding device 500 may apply the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform in a lossless system>. By applying any of these methods, the image encoding device 500 can obtain the same effect as that described in <Application of the present technology to an inverse quantization and inverse transform device> (that is, the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0277] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 500 described above will be described with reference to the flowchart of FIG.

[0278] When the image encoding process starts, in step S501, the reordering buffer 511 is controlled by the control unit 501 to reorder the frames of the input video data from the display order to the encoding order.

[0279] In step S502, the control unit 501 sets a processing unit for the input image held by the sorting buffer 511 (divides the image into blocks).

[0280] In step S503, the control unit 501 sets coding parameters for the input image held by the rearrangement buffer 511 (for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, etc.).

[0281] In step S504, the prediction unit 520 performs a prediction process to generate a predicted image, etc., of an optimal prediction mode. For example, in this prediction process, the prediction unit 520 performs intra prediction to generate a predicted image, etc., of an optimal intra prediction mode, performs inter prediction to generate a predicted image, etc., of an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value, etc.

[0282] In step S505, the calculation unit 512 calculates the difference between the input image and the prediction image of the optimal mode selected by the prediction process in step S504. That is, the calculation unit 512 derives residual data D between the input image and the prediction image. The residual data D derived in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to the case where the image is encoded as it is.

[0283] In step S506, the transform / quantization unit 513 performs a transform / quantization process on the residual data D derived by the process in step S505 using the encoding parameters such as the transform information Tinfo set in step S503, to derive a quantization coefficient level.

[0284] In step S507, the inverse quantization and inverse transform unit 516 performs inverse quantization and inverse transform processing on the quantization coefficient level derived in step S506 using the encoding parameters such as the transformation information Tinfo set in step S503, to derive residual data D'. This inverse quantization and inverse transform processing is the inverse processing of the transform quantization processing in step S506, and is the same processing as the inverse quantization and inverse transform processing in step S403 of the image decoding processing in FIG. 7.

[0285] In step S508, the calculation unit 517 generates a locally decoded decoded image by adding the predicted image generated by the prediction process in step S504 to the residual data D' derived by the inverse quantization and inverse transform process in step S507.

[0286] In step S509, the in-loop filter unit 518 performs in-loop filtering on the locally decoded image derived in the processing of step S508.

[0287] In step S510, the frame memory 519 stores the locally decoded image derived by the process in step S508 and the locally decoded image filtered in step S509.

[0288] In step S511, the encoding unit 514 encodes the quantization coefficient level derived by the transform and quantization process in step S506 to derive encoded data. At this time, the encoding unit 514 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo). Furthermore, the encoding unit 514 derives residual information RInfo from the quantization coefficient level, and encodes the residual information RInfo.

[0289] In step S512, the accumulation buffer 515 accumulates the encoded data derived in this manner, and outputs it to the outside of the image encoding device 500, for example as a bit stream. This bit stream is transmitted to a decoding side device (for example, the image decoding device 400) via, for example, a transmission path or a recording medium. In addition, the rate control unit 521 controls the rate as necessary. When the process of step S512 ends, the image encoding process ends.

[0290] <Flow of conversion and quantization process> Next, an example of the flow of the conversion and quantization process executed in step S506 in FIG. 11 will be described with reference to the flowchart in FIG.

[0291] When the conversion and quantization process is started, in step S541, the adaptive color conversion unit 541 adaptively converts the coefficient data res_x derived by the process of step S505 based on the cu_act_enabled_flag set in step S503 (FIG. 11), to derive coefficient data res_x'.

[0292] In step S542, the orthogonal transform unit 542 performs orthogonal transform on the coefficient data res_x' derived in step S541 using the transform information Tinfo, prediction mode information Pinfo, and the like set in step S503 (FIG. 11), to derive an orthogonal transform coefficient coef_x.

[0293] In step S543, the quantization unit 543 quantizes the orthogonal transform coefficient coef_x derived in step S542 using the transformation information Tinfo etc. set in step S503 (FIG. 11) to derive the quantization coefficient qcoef_x.

[0294] When the process of step S543 ends, the process returns to FIG.

[0295] <Adaptive color conversion process flow> Next, an example of the flow of the adaptive color conversion process executed in step S541 in FIG. 12 will be described with reference to the flowchart in FIG.

[0296] When the adaptive color conversion process is started, the selection unit 571 of the adaptive color conversion unit 541 determines in step S571 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S572.

[0297] In step S572, the YCgCo-R conversion unit 572 performs YCgCo-R conversion on the three-component coefficient data res_x to derive coefficient data res_x' after adaptive color conversion.

[0298] In step S573, the YCgCo-R conversion unit 572 supplies the derived coefficient data res_x' to the orthogonal transformation unit 542. When the process of step S573 ends, the adaptive color conversion process ends.

[0299] Also, if it is determined in step S571 that cu_act_enabled_flag is false, the process proceeds to step S574.

[0300] In step S574, the selection unit 571 supplies the coefficient data res_x as coefficient data res_x' after adaptive color transformation to the orthogonal transformation unit 542. When the process of step S574 ends, the adaptive color transformation process ends.

[0301] <Application of this technology to image encoding processing> In such an image encoding process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. That is, as the inverse quantization and inverse transform process of step S507, the inverse quantization and inverse transform process described with reference to the flowchart in FIG. 5 can be applied.

[0302] In this way, the inverse quantization and inverse transform unit 516 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform unit 516 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform.

[0303] That is to say, by performing the image coding process as described above, the image coding device 500 can suppress an increase in the load of the image coding process while suppressing a decrease in the quality of the decoded image.

[0304] In this image coding process (inverse quantization and inverse transform process (step S507)), the various methods of the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an image coding device> (i.e., the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0305] <6. Notes> <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0306] FIG. 14 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0307] In a computer 800 shown in FIG. 14, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0308] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0309] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0310] In the computer configured as above, the series of processes described above are executed by the CPU 801, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data and the like necessary for the CPU 801 to execute various processes as appropriate.

[0311] The program executed by the computer can be applied by recording it on a removable medium 821 such as a package medium, for example. In this case, the program can be installed in the storage unit 813 via the input / output interface 810 by mounting the removable medium 821 on the drive 815.

[0312] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0313] In addition, this program can also be installed in advance in the ROM 802 or the storage unit 813.

[0314] <Target of application of this technology> This technology can be applied to any image encoding or decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding and decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. In addition, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0315] The present technology can also be applied to a multi-view image encoding system that encodes a multi-view image including images from a plurality of views. The present technology can also be applied to a multi-view image decoding system that decodes encoded data of a multi-view image including images from a plurality of views. In this case, the present technology may be applied in encoding and decoding of each view.

[0316] Furthermore, the present technology can be applied to a hierarchical image coding (scalable coding) system that codes a hierarchical image that is multi-layered (hierarchical) so as to have a scalability function for a predetermined parameter. Also, the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes coded data of a hierarchical image that is multi-layered (hierarchical) so as to have a scalability function for a predetermined parameter. In that case, the present technology may be applied in the coding and decoding of each layer.

[0317] In addition, although the above describes the inverse adaptive color transformation device 100, the inverse quantization and inverse transformation device 200, the image decoding device 400, and the image encoding device 500 as application examples of the present technology, the present technology can be applied to any configuration.

[0318] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0319] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0320] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0321] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0322] <Fields and uses where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.

[0323] For example, the present technology can be applied to systems and devices used for providing ornamental content, etc. Furthermore, for example, the present technology can be applied to systems and devices used for traffic, such as monitoring traffic conditions and automatic driving control. Furthermore, for example, the present technology can be applied to systems and devices used for security. Furthermore, for example, the present technology can be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can be applied to systems and devices that monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife, etc. Furthermore, for example, the present technology can be applied to systems and devices used for sports.

[0324] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.

[0325] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.

[0326] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0327] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0328] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0329] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0330] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0331] Furthermore, the program executed by the computer may have the following features. For example, the processing of the steps of writing the program may be executed in chronological order according to the order described in this specification. The processing of the steps of writing the program may be executed in parallel. Furthermore, the processing of the steps of writing the program may be executed individually at a necessary timing, such as when called. In other words, the processing of each step may be executed in an order different from the order described above, so long as no contradiction occurs. Furthermore, the processing of the steps of writing the program may be executed in parallel with the processing of another program. Furthermore, the processing of the steps of writing the program may be executed in combination with the processing of another program.

[0332] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0333] The present technology can also be configured as follows. (1) a clipping processor that clips coefficient data that has been losslessly adaptively color transformed at a level based on a bit depth of the coefficient data; an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped at the level by the clip processing unit in the lossless manner; An image processing device comprising: (2) The clipping processing unit clips the luminance component and the chrominance component of the coefficient data at the same level. An image processing device as described in (1). (3) The clip processing unit clipping the luminance component and the chrominance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being an exponent obtained by adding 1 to the bit depth; The luminance component and the chrominance component of the coefficient data are clipped using a lower limit value obtained by multiplying a power of 2, the power being the value obtained by adding 1 to the bit depth, by -1. An image processing device as described in (2). (4) The level is a value based on the bit depth and the dynamic range of a buffer that stores the coefficient data. An image processing device as described in (2). (5) The level is a value derived using the smaller of a value based on the bit depth and not based on the dynamic range of the buffer, and a value not based on the bit depth and based on the dynamic range of the buffer. An image processing device according to (4). (6) The clip processing unit clipping the luminance component and the chrominance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer; The luminance component and the color component of the coefficient data are clipped to a lower limit value obtained by multiplying a power of 2, whichever is smaller, of a value obtained by adding 1 to the bit depth or a value obtained by subtracting 1 from the dynamic range of the buffer, by −1. An image processing device as described in (5). (7) The clip processing unit clipping a luminance component of the coefficient data at a first level; Clipping the color components of the coefficient data at a second level. An image processing device as described in (1). (8) The second level has a larger difference between an upper limit value and a lower limit value than the first level. An image processing device according to (7). (9) The clip processing unit clipping the luminance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2 with the bit depth as an exponent; clipping the luminance component of the coefficient data to a lower limit value obtained by multiplying a power of 2, with the bit depth as an exponent, by −1; clipping the color components of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the value obtained by adding 1 to the bit depth; The color components of the coefficient data are clipped using a lower limit value obtained by multiplying a power of 2, which is an exponent obtained by adding 1 to the bit depth, by −1. An image processing device according to (8). (10) The first level and the second level are values ​​based on the bit depth and a dynamic range of a buffer that stores the coefficient data. An image processing device according to (7). (11) The clip processing unit clipping the luminance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of the bit depth and the dynamic range of the buffer; clipping the luminance component of the coefficient data to a lower limit value obtained by multiplying a power of 2, whichever is smaller, of the bit depth or the dynamic range of the buffer minus 1, by −1; clipping the color components of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer; The color components of the coefficient data are clipped to a lower limit value obtained by multiplying a power of 2, whichever is smaller, of a value obtained by adding 1 to the bit depth or a value obtained by subtracting 1 from the dynamic range of the buffer, by −1. The image processing device according to (10). (12) The level is a value based on the dynamic range of a buffer that stores the coefficient data. An image processing device as described in (1). (13) The clip processing unit clipping the luminance and chrominance components of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being an exponent obtained by subtracting 1 from the dynamic range of the buffer; The luminance component and the color component of the coefficient data are clipped to a lower limit value obtained by multiplying a power of 2 obtained by subtracting 1 from the dynamic range of the buffer and multiplying the power of 2 by -1. An image processing device according to (12). (14) An inverse orthogonal transform unit that performs inverse orthogonal transform on orthogonal transform coefficients to generate the coefficient data that has been adaptively color transformed in the lossless manner, The clipping unit is configured to clip the coefficient data generated by the inverse orthogonal transform unit at the level. An image processing device as described in (1). (15) The image processing apparatus further includes an inverse quantization unit that inverse quantizes the quantized coefficients and generates the orthogonal transform coefficients, The inverse orthogonal transform unit is configured to perform inverse orthogonal transform on the orthogonal transform coefficients generated by the inverse quantization unit. An image processing device according to (14). (16) A decoding unit that decodes encoded data and generates the quantized coefficients, The inverse quantization unit is configured to inverse quantize the quantized coefficients generated by the decoding unit. An image processing device according to (15). (17) The inverse adaptive color transform unit is configured to inversely adaptively transform the coefficient data clipped at the level by the clip processing unit in the lossless manner to generate prediction residual data of image data; The method further includes a calculation unit that adds prediction data of the image data to the prediction residual data generated by the inverse adaptive color conversion unit to generate the image data. An image processing device according to (16). (18) A transform and quantization unit that performs adaptive color transform on image data in a lossless manner to generate the coefficient data, performs orthogonal transform on the coefficient data to generate the orthogonal transform coefficients, and quantizes the orthogonal transform coefficients to generate the quantized coefficients; a coding unit that generates coded data by coding the quantized coefficients generated by the transform and quantization unit; Further equipped with The inverse quantization unit is configured to inverse quantize the quantized coefficients generated by the transform and quantization unit. An image processing device according to (15). (19) A calculation unit that subtracts prediction data of the image data from the image data to generate prediction residual data, The transform and quantization unit is configured to perform adaptive color transform on the prediction residual data generated by the calculation unit in a lossless manner to generate the coefficient data, to perform orthogonal transform on the coefficient data to generate the orthogonal transform coefficients, and to quantize the orthogonal transform coefficients to generate the quantized coefficients. An image processing device according to (18). (20) clipping the lossless adaptive color transformed coefficient data at a level based on a bit depth of the coefficient data; Inversely adaptively color transforming the level-clipped coefficient data in the lossless manner. Image processing methods. [Explanation of symbols]

[0334] 100 inverse adaptive color transform device, 101 selection unit, 102 clip processing unit, 103 inverse YCgCo-R transform unit, 200 inverse quantization inverse transform device, 201 inverse quantization unit, 202 inverse orthogonal transform unit, 203 inverse adaptive color transform unit, 400 image decoding device, 401 control unit, 412 decoding unit, 413 inverse quantization inverse transform unit, 414 calculation unit, 415 in-loop filter unit, 416 sorting buffer, 417 frame memory, 418 prediction unit, 500 image encoding device, 501 control unit, 511 sorting buffer, 512 calculation unit, 313 transform quantization unit, 514 encoding unit, 515 accumulation buffer, 516 inverse quantization inverse transform unit, 517 calculation unit, 518 In-loop filter unit, 519 frame memory, 520 prediction unit, 521 rate control unit

Claims

1. A decoding unit that decodes encoded data and generates quantized coefficients; an inverse quantization unit that inversely quantizes the quantized coefficients generated by the decoding unit to generate orthogonal transform coefficients; an inverse orthogonal transform unit that performs inverse orthogonal transform on the orthogonal transform coefficients generated by the inverse quantization unit to generate coefficient data that has been adaptively color transformed in a lossless manner; a clipping processing unit that clips the coefficient data generated by the inverse orthogonal transform unit to an upper limit value obtained by subtracting 1 from a power of 2, the power being an exponent of a value obtained by adding 1 to a bit depth of the coefficient data, and a lower limit value obtained by multiplying a power of 2, the exponent being a value obtained by adding 1 to the bit depth, by −1; an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value by the clip processing unit in the lossless manner; An image processing device comprising:

2. The clipping processing unit clips the luminance component and the color component of the coefficient data at the same upper limit value and the same lower limit value. The image processing device according to claim 1 .

3. The inverse adaptive color transform unit is configured to inversely adaptively transform the coefficient data clipped at the upper limit value and the lower limit value by the clip processing unit in the reversible manner to generate prediction residual data of image data; The method further includes a calculation unit that adds prediction data of the image data to the prediction residual data generated by the inverse adaptive color conversion unit to generate the image data. The image processing device according to claim 1 .

4. A method for decoding encoded data and generating quantized coefficients, The generated quantized coefficients are inversely quantized to generate orthogonal transform coefficients; performing an inverse orthogonal transform on the generated orthogonal transform coefficients to generate coefficient data that has been adaptively color transformed in a lossless manner; The generated coefficient data is clipped to an upper limit value obtained by subtracting 1 from a power of 2 having an exponent of the bit depth of the coefficient data plus 1, and to a lower limit value obtained by multiplying a power of 2 having an exponent of the bit depth plus 1 by −1; The coefficient data clipped at the upper limit value and the lower limit value is inversely adaptively color-transformed in the lossless manner. Image processing methods.

5. A transform and quantization unit that performs adaptive color transform on image data in a lossless manner to generate coefficient data, performs orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantizes the orthogonal transform coefficients to generate quantized coefficients; a coding unit that generates coded data by coding the quantized coefficients generated by the transform and quantization unit; an inverse quantization unit that inversely quantizes the quantized coefficients generated by the transform and quantization unit to generate the orthogonal transform coefficients; an inverse orthogonal transform unit that performs inverse orthogonal transform on the orthogonal transform coefficients generated by the inverse quantization unit to generate the coefficient data that has been adaptively color transformed in the lossless manner; a clipping processing unit that clips the coefficient data generated by the inverse orthogonal transform unit to an upper limit value obtained by subtracting 1 from a power of 2, the power being an exponent of a value obtained by adding 1 to a bit depth of the coefficient data, and a lower limit value obtained by multiplying a power of 2, the exponent being a value obtained by adding 1 to the bit depth, by −1; an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value by the clip processing unit in the lossless manner; An image processing device comprising:

6. The clipping processing unit clips the luminance component and the color component of the coefficient data at the same upper limit value and the same lower limit value. The image processing device according to claim 5 .

7. A calculation unit that subtracts prediction data of the image data from the image data and generates prediction residual data, The transform and quantization unit is configured to perform adaptive color transform on the prediction residual data generated by the calculation unit in the reversible manner to generate the coefficient data, to perform orthogonal transform on the coefficient data to generate the orthogonal transform coefficients, and to quantize the orthogonal transform coefficients to generate the quantized coefficients. The image processing device according to claim 5 .

8. A method for generating coefficient data by performing adaptive color transformation on image data in a lossless manner, performing an orthogonal transformation on the coefficient data to generate orthogonal transform coefficients, and quantizing the orthogonal transform coefficients to generate quantized coefficients; encoding the generated quantized coefficients to generate encoded data; The generated quantized coefficients are inversely quantized to generate the orthogonal transform coefficients; performing an inverse orthogonal transform on the generated orthogonal transform coefficients to generate the coefficient data that has been adaptively color transformed in the lossless manner; The generated coefficient data is clipped to an upper limit value obtained by subtracting 1 from a power of 2 having an exponent of the bit depth of the coefficient data plus 1, and to a lower limit value obtained by multiplying a power of 2 having an exponent of the bit depth plus 1 by −1; The coefficient data clipped at the upper limit value and the lower limit value is inversely adaptively color-transformed in the lossless manner. Image processing methods.

Images