Image processing device and method

Abstract

To suppress an increase in the load of inverse adaptive color transformation processing while suppressing an increase in the distortion of coefficient data after inverse adaptive color transformation.SOLUTION: Coefficient data adaptively color transformed in a lossless manner is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at that level is inversely adaptively color transformed in a lossless manner. The present disclosure is applicable to, for example, an image processing device, an image encoding device, an image decoding device, a transmission device, a reception device, a transmission / reception device, an information processing device, an imaging device, a playback device, an electronic device, an image processing method, an information processing method, and the like.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method that can suppress an increase in the load of inverse adaptive color transformation processing while suppressing an increase in distortion of coefficient data after inverse adaptive color transformation. [Background technology]

[0002] Conventionally, coding methods have been proposed in which prediction residuals of moving images are derived, coefficients are transformed, and quantized for coding (see, for example, Non-Patent Document 1 and Non-Patent Document 2). Also, adaptive color transform (ACT) has been proposed as a coding tool to improve coding efficiency in RGB444, which transforms coefficient data in the RGB domain into coefficient data in the YCgCo domain. Furthermore, a reversible method for this adaptive color transform has been proposed (see, for example, Non-Patent Document 3).

[0003] Incidentally, in lossy inverse adaptive color transformation (Inverse ACT), in order to suppress an increase in the load of the inverse adaptive color transformation process, it has been proposed to clip the coefficient data of the YCgCo domain, which is the input signal, at [-2^bitDepth, 2^bitDepth-1] (see, for example, Non-Patent Document 4). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, "Versatile Video Coding (Draft 7)", JVET-P2001-vE, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Feneva, CH, 1-11 Oct 2019

Non-Patent Document 2

Non-Patent Document 3

Non-Patent Document 4

Summary of the Invention

[0005] However, the range of values ​​of coefficient data in the YCgCo domain after lossless adaptive color conversion is wider than the range of values ​​of coefficient data in the YCgCo domain after lossy adaptive color conversion. Therefore, if the method described in Non-Patent Document 4 is applied to the lossless inverse adaptive color conversion disclosed in Non-Patent Document 3, the coefficient data in the YCgCo domain may change due to clipping, which may increase distortion of the coefficient data in the RGB domain after the inverse adaptive color conversion.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of coefficient data after inverse adaptive color conversion. [Means for solving the problem]

[0007] An image processing device according to one aspect of the present technology is an image processing device including: a clipping processing unit that clips coefficient data that has been adaptively color transformed in a reversible manner to an upper limit value obtained by subtracting 1 from a power of 2 where the power value is the value obtained by adding 1 to the bit depth of the coefficient data, and a lower limit value obtained by multiplying the power of 2 where the power value is the value obtained by adding 1 to the bit depth and -1; and an inverse adaptive color conversion unit that performs inverse adaptive color conversion in the reversible manner on the coefficient data clipped by the clipping processing unit at the upper limit value and the lower limit value.

[0008] An image processing method according to one aspect of the present technology includes clipping coefficient data that has been adaptively color transformed in a lossless manner to an upper limit value obtained by subtracting 1 from a power of 2, where the power value is the value obtained by adding 1 to the bit depth of the coefficient data, and a lower limit value obtained by multiplying the power of 2, where the power value is the value obtained by adding 1 to the bit depth, by −1; and performing inverse adaptive color transformation on the coefficient data clipped at the upper limit value and the lower limit value in a lossless manner.

[0009] In an image processing device and method according to one aspect of the present technology, coefficient data that has been adaptively color transformed in a reversible manner is clipped to an upper limit value obtained by subtracting 1 from a power of 2 where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data, and a lower limit value obtained by multiplying the power of 2 where the exponent is the value obtained by adding 1 to the bit depth and -1, and the coefficient data clipped at the upper and lower limit values ​​is then subjected to an inverse adaptive color transformation in a reversible manner. [Brief explanation of the drawings]

[0010] [Figure 1] 10A and 10B are diagrams illustrating an example of the difference in range between YCgCo conversion and YCgCo-R conversion. [Figure 2] 1 is a block diagram showing an example of the main configuration of an inverse adaptive color conversion device. [Figure 3] 10 is a flowchart illustrating an example of the flow of an inverse adaptive color conversion process. [Figure 4] FIG. 1 is a block diagram illustrating an example of the main configuration of an inverse quantization and inverse transform device. [Figure 5] 10 is a flowchart illustrating an example of the flow of an inverse quantization and inverse transform process. [Figure 6] FIG. 1 is a block diagram illustrating an example of the main configuration of an image decoding device. [Figure 7] 10 is a flowchart showing an example of the flow of an image decoding process. [Figure 8] FIG. 1 is a block diagram illustrating an example of the main configuration of an image encoding device. [Figure 9] FIG. 2 is a block diagram illustrating an example of the main configuration of a transform and quantization unit. [Figure 10] FIG. 2 is a block diagram illustrating an example of the main configuration of an adaptive color conversion unit. [Figure 11] 10 is a flowchart illustrating an example of the flow of an image encoding process. [Figure 12] 10 is a flowchart illustrating an example of the flow of a transform and quantization process. [Figure 13] 10 is a flowchart illustrating an example of the flow of adaptive color conversion processing. [Figure 14] FIG. 1 is a block diagram illustrating an example of the main configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0011] Modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described below in the following order. 1. Clipping of lossless inverse adaptive color transform 2. First embodiment (reverse adaptive color conversion device) 3. Second embodiment (inverse quantization and inverse transform device) 4. Third Embodiment (Image Decoding Apparatus) 5. Fourth embodiment (image encoding device) 6. Supplementary Notes

[0012] <1. Clipping process for lossless inverse adaptive color transformation> <References that support the technical content and terminology> The scope of disclosure of the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents that were publicly known at the time of filing, as well as the contents of other documents referenced in the following non-patent documents.

[0013] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: (mentioned above) Non-patent document 4: (mentioned above) Non-Patent Document 5: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-Patent Document 6: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0014] In other words, the contents of the above-mentioned non-patent documents are also used as a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above-mentioned non-patent documents are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and meet the support requirements of the claims. Similarly, even if technical terms such as parsing, syntax, and semantics are not directly described in the examples, they are considered to be within the scope of the disclosure of the present technology and meet the support requirements of the claims.

[0015] Furthermore, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) refers to any partial region within a picture, and its size, shape, characteristics, etc. are not limited unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), sub-block, macroblock, tile, or slice, as described in the above-mentioned non-patent document.

[0016] Furthermore, when specifying such block sizes, the block sizes may be specified not only directly but also indirectly. For example, the block sizes may be specified using identification information for identifying the sizes. Furthermore, for example, the block sizes may be specified by the ratio or difference with respect to the size of a reference block (e.g., LCU, SCU, etc.). For example, when transmitting information specifying the block size as a syntax element, the information indirectly specifying the size as described above may be used as the information. This may reduce the amount of information and improve coding efficiency. Furthermore, the specification of the block sizes may also include specification of a range of block sizes (e.g., specification of a range of allowable block sizes, etc.).

[0017] Furthermore, in this specification, "encoding" refers not only to the overall process of converting an image into a bitstream, but also to some of the processes. For example, it not only includes processes that encompass prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also includes a process that collectively refers to quantization and arithmetic coding, a process that encompasses prediction processing, quantization, and arithmetic coding, etc. Similarly, "decoding" refers not only to the overall process of converting a bitstream into an image, but also to some of the processes. For example, it not only includes processes that encompass inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also includes a process that encompasses inverse arithmetic decoding and inverse quantization, a process that encompasses inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0018] <Adaptive color conversion> As a coding tool for improving coding efficiency in RGB444, adaptive color transform (ACT), which converts coefficient data in the RGB domain into coefficient data in the YCgCo domain, has been proposed. This adaptive color transform is also called YCgCo transform. The inverse process of adaptive color transform is also called inverse adaptive color transform (Inverse ACT). The inverse process of YCgCo transform is also called inverse YCgCo transform. This YCgCo transform (inverse YCgCo transform) is a lossy method. Therefore, in YCgCo transform (inverse YCgCo transform), the conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain can be lossy (lossy). In other words, by converting RGB domain coefficient data into YCgCo and then performing inverse YCgCo transform, it can be difficult to generate (restore) RGB domain coefficient data that is identical to the RGB domain coefficient data before the YCgCo transform.

[0019] In response to this, Non-Patent Document 3 proposed a lossless method for adaptive color conversion. This lossless adaptive color conversion is also called YCgCo-R conversion. The inverse process of YCgCo-R conversion is also called inverse YCgCo-R conversion. In other words, this YCgCo-R conversion (inverse YCgCo-R conversion) is a lossless method. Therefore, YCgCo-R conversion (inverse YCgCo-R conversion) can achieve lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain. In other words, by performing YCgCo-R conversion on RGB domain coefficient data and then inverse YCgCo-R conversion, it is possible to generate (restore) RGB domain coefficient data that is identical to the RGB domain coefficient data before YCgCo-R conversion.

[0020] Therefore, when applying adaptive color transform and inverse adaptive color transform, it may be difficult to achieve lossless coding if lossy YCgCo transform and inverse YCgCo transform are applied. In contrast, lossless coding can be achieved by applying YCgCo-R transform and inverse YCgCo-R transform.

[0021] <CgCo-R Conversion and Inverse YCgCo-R Conversion> The YCgCo-R conversion converts RGB components to YCgCo components as shown in the following equations (1) to (4).

[0022] TIFF0007747236000001.tif832···(1) TIFF0007747236000002.tif1038···(2) TIFF0007747236000003.tif1032···(3) TIFF0007747236000004.tif1247···(4)

[0023] Through the YCgCo-R conversion as shown in equations (1) to (4), similar to the case of YCgCo conversion, the coefficient data in the RGB domain can be converted to the coefficient data in the YCgCo domain equivalent to the YCbCr domain only by simple shift operations and addition and subtraction. The inverse YCgCo-R conversion is performed as shown in the following equations (5) to (8).

[0024] TIFF0007747236000005.tif1040···(5) TIFF0007747236000006.tif1031···(6) TIFF0007747236000007.tif1041···(7) TIFF0007747236000008.tif1037···(8)

[0025] <Clipping Process> Incidentally, Non-Patent Document 4 proposes clipping coefficient data in the YCgCo domain, which is the input signal, at [-2^bitDepth, 2^bitDepth-1] in order to suppress an increase in the load of the inverse adaptive color conversion process in lossy inverse adaptive color conversion (Inverse ACT). Note that [A, B] indicates a range with A as the lower limit and B as the upper limit. Also, "clipping at [A, B]" means that values ​​equal to or less than the lower limit A of the input signal are set to A, and values ​​equal to or greater than the upper limit B are set to B. BitDepth indicates the bit depth of the coefficient data in the RGB domain before adaptive color conversion.

[0026] For example, the range of coefficient data for bit depth bitDepth is [-2^bitDepth, 2^bitDepth-1]. When RGB domain coefficient data with this range is converted to YCgCo, the range of coefficient data for each component in the YCgCo domain is theoretically within the range of [-2^bitDepth, 2^bitDepth-1]. However, in reality, due to the influence of some external factor, the coefficient data for each component in the YCgCo domain may take a value outside this range. In inverse adaptive color conversion processing, if the range of input signals is expanded in this way, there is a risk of an increase in the load of the inverse adaptive color conversion processing. In particular, there is a risk of increased costs when implementing it in hardware.

[0027] Therefore, the coefficient data of each component in the YCgCo domain, which is the input signal for the inverse YCgCo conversion, is clipped at [-2^bitDepth, 2^bitDepth-1]. Y [x][y], r Cb [x][y], r Cr [x][y] is processed as shown in the following equations (9) to (11).

[0028] TIFF0007747236000009.tif10132...(9) TIFF0007747236000010.tif9131...(10) TIFF0007747236000011.tif9132...(11)

[0029] r Y [x][y] indicates the coefficient data of the Y component. Cb [x][y] indicates the coefficient data of the Cg component. Cr [x][y] indicate the coefficient data of the Co component. Clip3(A,B,C) indicates a clip function that clips C with lower limit A and upper limit B. << indicates a bit shift (i.e., a power of 2).

[0030] By doing this, it becomes unnecessary to consider unnecessary processing levels outside this range during inverse YCgCo conversion. This makes it possible to suppress an increase in the load of inverse adaptive color conversion processing. In particular, when implementing in hardware, it is possible to suppress an increase in cost.

[0031] The coefficient data thus clipped is then subjected to a lossless inverse YCgCo-R transform as described in Non-Patent Document 3. In this case, the coefficient data r processed as shown in Equations (9) to (11) is Y [x][y], r Cb [x][y], r Cr [x][y] are processed as shown in the following equations (12) to (15).

[0032] TIFF0007747236000012.tif1285...(12) TIFF0007747236000013.tif1275...(13) TIFF0007747236000014.tif1288...(14) TIFF0007747236000015.tif1261...(15)

[0033] It should be noted that the formulas (12) to (15) are equivalent to the above formulas (5) to (8).

[0034] <Clip processing in lossless adaptive color transformation> However, in the case of the lossless YCgCo-R conversion described above, the value range of the coefficient data after conversion is wider than that in the case of YCgCo conversion. For example, when coefficient data with a bit depth of bitDepth in the RGB domain is converted into YCgCo-R, the value range of the coefficient data of the Y component, which is the luminance component, is [-2^bitDepth, 2^bitDepth-1]. Furthermore, the value range of the coefficient data of the Cg component and Co component, which are chrominance components, is [-2^(bitDepth+1), 2^(bitDepth+1)-1]. In other words, the dynamic range of the coefficient data of the Y component is bitDepth+1, and the dynamic range of the coefficient data of the Cg component and Co component is bitDepth+2.

[0035] Therefore, when the method described in Non-Patent Document 4 is applied to the lossless adaptive color transform disclosed in Non-Patent Document 3, there is a risk that the coefficient data in the YCgCo domain will change due to clipping, and that the distortion of the coefficient data in the RGB domain after the inverse adaptive color transform will increase. Note that this "distortion" refers to the mismatch with the coefficient data in the RGB domain before the inverse adaptive color transform, i.e., the difference between the coefficient data in the RGB domain before and after the inverse adaptive color transform. In other words, with this method, there is a risk that it will be difficult to achieve lossless transformation between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain.

[0036] 1 is a diagram showing an example of the relationship between input and output values ​​when a residual signal in the RGB domain is converted into YCgCo or YCgCo-R. The bit depth of the RGB domain coefficient data, which is the input for the YCgCo and YCgCo-R conversions, is assumed to be 10 bits (bitDepth=10). The value range of this RGB domain coefficient data is assumed to be [-2^bitDepth, 2^bitDepth-1].

[0037] A in Figure 1 shows an example of the relationship between the G component (input value) and the Y component (output value). B in Figure 1 shows an example of the relationship between the B component (input value) and the Cg component (output value). C in Figure 1 shows an example of the relationship between the R component (input value) and the Co component (output value).

[0038] In addition, in Fig. 1, gray circles indicate an example of the relationship between input values ​​and output values ​​in the case of YCgCo conversion. White circles indicate an example of the relationship between input values ​​and output values ​​in the case of YCgCo-R conversion. The thick solid lines indicate an example of the upper clip limit. The thick dotted lines indicate an example of the lower clip limit. Here, the upper and lower limit values ​​of the clip processing described in Non-Patent Document 4 are shown.

[0039] As shown in Figures 1B and 1C, it can be seen that the coefficient data of the Cg component and the coefficient data of the Co component after YCgCo-R conversion may exceed the upper and lower clipping limits. In such cases, clipping processing clips values ​​above the upper limit to the upper limit, and values ​​below the lower limit to the lower limit. In other words, the values ​​of some of the coefficient data change. This could result in increased distortion (mismatch with the data before YCgCo-R conversion) in the coefficient data after inverse YCgCo-R conversion.

[0040] Since the inverse adaptive color transform becomes irreversible, it may be difficult to achieve lossless coding when applying the inverse adaptive color transform. Furthermore, the distortion of the RGB domain coefficient data after the inverse adaptive color transform increases, which may increase the difference between the decoded image and the image before encoding, resulting in a decrease in the quality of the decoded image.

[0041] <Range setting for clipping processing for lossless adaptive color transformation> Therefore, the coefficient data that has been adaptively color transformed using the lossless method, which is the input to the lossless inverse adaptive color transformation, is clipped at a level based on the bit depth of the coefficient data.

[0042] For example, in an information processing method, coefficient data that has been adaptively color transformed in a reversible manner is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at that level is subjected to inverse adaptive color transformation in a reversible manner.

[0043] For example, an information processing device may be provided with a clipping processing unit that clips coefficient data that has been adaptively color converted in a reversible manner at a level based on the bit depth of the coefficient data, and an inverse adaptive color conversion unit that inversely adaptively color converts the coefficient data clipped at that level by the clipping processing unit in a reversible manner.

[0044] By doing so, it is possible to clip the input of the lossless inverse adaptive color transform at a level that does not increase the distortion of the coefficient data after the inverse adaptive color transform. Therefore, it is possible to suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in the distortion of the coefficient data after the inverse adaptive color transform.

[0045] In particular, by setting the upper and lower limit values ​​of the clipping process so that the range between the upper and lower limit values ​​includes the theoretical range of values ​​that coefficient data adaptively color converted using a lossless method can take (i.e., so that the range is wider than the theoretical range of values ​​that the coefficient data can take), it is possible to suppress distortion of the coefficient data after the inverse adaptive color conversion.In other words, it is possible to suppress an increase in the load of the inverse adaptive color conversion process while achieving lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain.

[0046] <Method 1> During the clipping process, the luminance component and the color components (color difference components) of the coefficient data adaptively color converted in a lossless manner may be clipped at the same level. The coefficient data adaptively color converted in a lossless manner is composed of a Y component, which is a luminance component, a Cg component, which is a color component (color difference component), and a Co component, which is a color component (color difference component). In other words, all of these components may be clipped at the same level. This makes the clipping process easier and reduces the load compared to clipping at different levels for each component.

[0047] <Method 1-1> For example, the luminance component and color component (color difference component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2, where the power value is the value obtained by adding 1 to the bit depth of the coefficient data that has been adaptively color converted in a lossless manner. Alternatively, the luminance component and color component (color difference component) of the coefficient data may be clipped using a lower limit value obtained by multiplying the power of 2, where the power value is the value obtained by adding 1 to the bit depth of the coefficient data that has been adaptively color converted in a lossless manner, by -1. Furthermore, both the upper limit value and the lower limit value may be clipped.

[0048] For example, the upper limit actResMax of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (16) using a value obtained by adding 1 to the bit depth of the coefficient data. Also, the lower limit actResMin of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (17) using a value obtained by adding 1 to the bit depth of the coefficient data.

[0049] TIFF0007747236000016.tif978...(16) TIFF0007747236000017.tif1077...(17)

[0050] That is, the upper limit value actResMax is set to a value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth. Also, the lower limit value actResMin is set to a value obtained by multiplying the power of 2, where the exponent is the value obtained by adding 1 to the bit depth, by -1. Then, as shown in the following equations (18) to (20), the upper limit value actResMax and the lower limit value actResMin are used to clip the luminance component and chrominance component (chrominance component) of the coefficient data adaptively color converted using a lossless method.

[0051] TIFF0007747236000018.tif11115...(18) TIFF0007747236000019.tif10113...(19) TIFF0007747236000020.tif10112...(20)

[0052] Then, the coefficient data thus clipped is subjected to inverse YCgCo-R conversion as shown in the above equations (12) to (15), to derive coefficient data in the RGB domain.

[0053] By doing so, the range of values ​​between the upper and lower limit values ​​of the clipping process can be made wider than the range of values ​​that coefficient data adaptively color transformed using a lossless method can theoretically take. Therefore, by clipping at these upper and lower limit values, it is possible to suppress distortion of coefficient data after inverse adaptive color transformation. In other words, it is possible to suppress an increase in the load of the inverse adaptive color transformation process while realizing lossless transformation between coefficient data in the RGB domain and coefficient data in the YCgCo domain.

[0054] <Method 1-2> Furthermore, clipping may be performed taking into account the dynamic range of the buffer. For example, during clipping, coefficient data adaptively color transformed using a lossless method may be clipped at a level based on its bit depth and the dynamic range of the buffer that holds the coefficient data during inverse adaptive color transformation. By clipping coefficient data in this way while taking into account the dynamic range (hardware limitations) of the buffer, it is possible to prevent buffer overflow. In other words, since the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data to be held, it is possible to prevent an increase in the dynamic range of the buffer and therefore an increase in costs.

[0055] For example, the clipping level may be determined using the smaller of a value based on the bit depth of the coefficient data adaptively color transformed in a lossless manner but not based on the dynamic range of the buffer that stores the coefficient data, and a value based on the dynamic range of the buffer but not based on the bit depth. In this way, by setting the upper and lower limits using the smaller value (i.e., by clipping the coefficient data to a narrower value range), it is possible to satisfy the hardware limitations of the buffer and suppress an increase in the distortion of the coefficient data after the inverse adaptive color transformation, while suppressing an increase in the load of the inverse adaptive color transformation process.

[0056] For example, the luminance component and color component (chrominance component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted using a lossless method and the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data. Alternatively, the luminance component and color component (chrominance component) of the coefficient data may be clipped using a lower limit value obtained by multiplying -1 by a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted using a lossless method and the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data. Furthermore, both the upper limit clipping and the lower limit clipping may be performed.

[0057] For example, the upper limit actResMax of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (21) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data. Also, the lower limit actResMin of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (22) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data.

[0058] TIFF0007747236000021.tif10130...(21) TIFF0007747236000022.tif9119...(22)

[0059] That is, the upper limit value actResMax is set to a value obtained by subtracting 1 from a power of 2 where the exponent is the smaller of the value obtained by adding 1 to the bit depth or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data.The lower limit value actResMin is set to a value obtained by multiplying the smaller of the value obtained by adding 1 to the bit depth or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data by −1.The upper limit value actResMax and the lower limit value actResMin are then used to clip the luminance component and chrominance component (chrominance component) of the coefficient data that has been adaptively color converted using a lossless method, as shown in the above-mentioned equations (18) to (20).

[0060] Then, the coefficient data thus clipped is subjected to inverse YCgCo-R conversion as shown in the above equations (12) to (15), to derive coefficient data in the RGB domain.

[0061] By doing so, it is possible to satisfy the hardware limitations of the buffer, suppress an increase in the distortion of the coefficient data after the inverse adaptive color transformation, and suppress an increase in the load of the inverse adaptive color transformation process.

[0062] <Method 2> As described above in <Clipping in lossless adaptive color conversion>, the range of coefficient data for the Cg and Co components, which are color components (color difference components), is wider than the range of coefficient data for the Y component, which is a luminance component.

[0063] Therefore, the luminance component and the chrominance component of the coefficient data may be clipped at each level. For example, the luminance component of the coefficient data may be clipped at a first level, and the chrominance component of the coefficient data may be clipped at a second level. In this way, the coefficient data of each component can be clipped at a value range corresponding to that component, thereby further suppressing an increase in the load of the inverse adaptive color transformation process while suppressing an increase in the distortion of the coefficient data after the inverse adaptive color transformation.

[0064] For example, if clipping is performed according to a component with a wide range of values, the clipping width (between the upper and lower limits) for components with a narrower range will be unnecessarily wide, and the load of the inverse adaptive color conversion process will be reduced.As described above, by clipping each component with a width appropriate to that component, the load of the inverse adaptive color conversion process can be further reduced.

[0065] <Method 2-1> The difference between the upper and lower limit values ​​of the second level may be wider than the difference between the upper and lower limit values ​​of the first level. As described above, the value range of the coefficient data of the Cg and Co components, which are chrominance components, is wider than the value range of the coefficient data of the Y component, which is a luminance component. Therefore, by making the difference between the upper and lower limit values ​​of clipping for the coefficient data of the Cg and Co components, which are chrominance components, wider than the difference between the upper and lower limit values ​​of clipping for the coefficient data of the Y component, which is a luminance component, it is possible to further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data after the inverse adaptive color conversion.

[0066] For example, the luminance component of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2 where the bit depth of the coefficient data adaptively color converted in a lossless manner is used as the exponent. Furthermore, the luminance component of the coefficient data may be clipped using a lower limit value obtained by multiplying −1 by a power of 2 where the bit depth of the coefficient data adaptively color converted in a lossless manner is used as the exponent. Furthermore, both the upper limit value and the lower limit value may be clipped for the luminance component. Furthermore, the color component (chrominance component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2 where the bit depth of the coefficient data adaptively color converted in a lossless manner is used as the exponent. Furthermore, the color component (chrominance component) of the coefficient data may be clipped using a lower limit value obtained by multiplying −1 by a power of 2 where the bit depth of the coefficient data adaptively color converted in a lossless manner is used as the exponent. Alternatively, both the upper limit and lower limit clipping may be performed for the color components (chrominance components).Furthermore, both the upper limit and lower limit clipping may be performed for both the luminance component and the color components (chrominance components) as described above.

[0067] For example, the upper limit actResMaxY of the clipping process performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in equation (23) below, using the bit depth of the coefficient data. The lower limit actResMinY of the clipping process performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in equation (24) below, using the bit depth of the coefficient data. The upper limit actResMaxC of the clipping process performed on the color components (chrominance components) of coefficient data adaptively color converted in a lossless manner is set as shown in equation (25) below, using the bit depth of the coefficient data. The lower limit actResMinC of the clipping process performed on the color components (chrominance components) of coefficient data adaptively color converted in a lossless manner is set as shown in equation (26) below, using the bit depth of the coefficient data.

[0068] TIFF0007747236000023.tif878...(23) TIFF0007747236000024.tif873...(24) TIFF0007747236000025.tif1088...(25) TIFF0007747236000026.tif1085...(26)

[0069] That is, the upper limit value actResMaxY for the luminance component is set to a value obtained by subtracting 1 from a power of 2 where the bit depth is the exponent. Also, the lower limit value actResMinY for the luminance component is set to a value obtained by multiplying a power of 2 where the bit depth is the exponent by -1. Then, using these upper limit values ​​actResMaxY and lower limit values ​​actResMinY, the luminance component of the coefficient data that has been adaptively color converted using a lossless method is clipped, as shown in the following equation (27).

[0070] TIFF0007747236000027.tif10120...(27)

[0071] The upper limit actResMaxC for the color components (chrominance components) is set to a value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth. The lower limit actResMinC for the color components (chrominance components) is set to a value obtained by multiplying the power of 2, where the exponent is the value obtained by adding 1 to the bit depth, by -1. These upper limit actResMaxC and lower limit actResMinC are then used to clip the color components (chrominance components) of the coefficient data that has been adaptively color converted using a lossless method, as shown in the following equations (28) and (29).

[0072] TIFF0007747236000028.tif10121...(28) TIFF0007747236000029.tif11123...(29)

[0073] Then, the coefficient data thus clipped is subjected to inverse YCgCo-R conversion as shown in the above equations (12) to (15), to derive coefficient data in the RGB domain.

[0074] By doing this, the coefficient data of each component can be clipped within a value range corresponding to that component, thereby suppressing an increase in distortion of the coefficient data after the inverse adaptive color conversion, while further suppressing an increase in the load of the inverse adaptive color conversion process.

[0075] <Method 2-2> Furthermore, clipping may be performed taking into account the dynamic range of the buffer. For example, during clipping, coefficient data adaptively color-converted in a lossless manner may be clipped at a level based on the bit depth and the dynamic range of the buffer that stores the coefficient data during inverse adaptive color conversion. That is, the first and second levels may be values ​​based on the bit depth of the coefficient data adaptively color-converted in a lossless manner and the dynamic range of the buffer that stores the coefficient data, respectively. By clipping the coefficient data in this way while taking into account the dynamic range (hardware limitation) of the buffer, it is possible to prevent buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be stored, it is possible to prevent an increase in the dynamic range of the buffer and thus prevent an increase in costs.

[0076] For example, the level at which the luminance component is clipped may be determined based on the bit depth of coefficient data adaptively color transformed in a lossless manner, but not on the dynamic range of a buffer that stores the coefficient data, or a value that is not based on the bit depth and is based on the dynamic range of the buffer. The level at which the color components (chrominance components) are clipped may be determined based on the bit depth of coefficient data adaptively color transformed in a lossless manner plus 1, but not on the dynamic range of a buffer that stores the coefficient data, or a value that is not based on the bit depth and is based on the dynamic range of the buffer. By setting the upper and lower limits using the smaller value (i.e., by clipping the coefficient data to a narrower value range), it is possible to satisfy the hardware limitations of the buffer, suppress an increase in distortion of each component of the coefficient data after the inverse adaptive color transformation, and further suppress an increase in the load of the inverse adaptive color transformation process.

[0077] For example, the luminance component of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2 where the power is the smaller of the bit depth of the coefficient data adaptively color converted using a lossless method or the dynamic range of the buffer that stores the coefficient data. Alternatively, the luminance component of the coefficient data may be clipped using a lower limit value obtained by multiplying the power of 2 where the power is the smaller of the bit depth of the coefficient data adaptively color converted using a lossless method or the dynamic range of the buffer that stores the coefficient data by 1. Furthermore, both the upper limit clipping and the lower limit clipping may be performed on the luminance component.

[0078] Alternatively, the color components (chrominance components) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted using a lossless method and the value obtained by subtracting 1 from the dynamic range of the buffer storing the coefficient data. Furthermore, the color components (chrominance components) of the coefficient data may be clipped using a lower limit value obtained by multiplying −1 by a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data adaptively color converted using a lossless method and the value obtained by subtracting 1 from the dynamic range of the buffer storing the coefficient data. Furthermore, both the upper limit and lower limit clipping for the color components (chrominance components) may be performed. Furthermore, both the upper limit and lower limit clipping for the luminance component and the color components (chrominance components) may be performed as described above.

[0079] For example, the upper limit actResMaxY of the clipping process performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in equation (30) below, using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data. The lower limit actResMinY of the clipping process performed on the luminance component of coefficient data adaptively color converted in a lossless manner is set as shown in equation (31) below, using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data. The upper limit actResMaxC of the clipping process performed on the color component (chrominance component) of coefficient data adaptively color converted in a lossless manner is set as shown in equation (32) below, using the value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data. In addition, the lower limit actResMinC of the clipping process performed on the color components (chrominance components) of the coefficient data that has been adaptively color converted using a lossless method is set as shown in the following equation (33) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data.

[0080] TIFF0007747236000030.tif10133...(30) TIFF0007747236000031.tif11133...(31) TIFF0007747236000032.tif10141...(32) TIFF0007747236000033.tif11139...(33)

[0081] That is, the upper limit value actResMaxY for the luminance component is set to a value obtained by subtracting 1 from a power of 2 where the exponent is the smaller of the bit depth and the dynamic range of the buffer that stores the coefficient data. The lower limit value actResMinY for the luminance component is set to a value obtained by multiplying the power of 2 where the exponent is the smaller of the bit depth and the dynamic range of the buffer that stores the coefficient data by 1, by -1. The upper limit actResMaxY and lower limit actResMinY are then used to clip the luminance component of the coefficient data that has been adaptively color converted using a lossless method, as shown in the above equation (27).

[0082] The upper limit actResMaxC for the color components (chrominance components) is set to a value obtained by subtracting 1 from a power of 2 obtained by adding 1 to the bit depth or subtracting 1 from the dynamic range of the buffer that stores the coefficient data, whichever is smaller. The lower limit actResMinC for the color components (chrominance components) is set to a value obtained by multiplying -1 by a power of 2 obtained by adding 1 to the bit depth or subtracting 1 from the dynamic range of the buffer that stores the coefficient data. The color components (chrominance components) of the coefficient data that have been adaptively color converted using a lossless method are then clipped using these upper limit actResMaxC and lower limit actResMinC, as shown in the above equations (28) and (29).

[0083] Then, the coefficient data thus clipped is subjected to inverse YCgCo-R conversion as shown in the above equations (12) to (15), to derive coefficient data in the RGB domain.

[0084] By doing so, it is possible to satisfy the hardware limitations of the buffer, suppress an increase in distortion of each component of the coefficient data after the inverse adaptive color transformation, and further suppress an increase in the load of the inverse adaptive color transformation process.

[0085] <Method 3> During inverse adaptive color transformation, the coefficient data may be clipped at a level based on the dynamic range of the buffer that holds the coefficient data. By clipping the coefficient data in this way while taking into account the dynamic range (hardware limitations) of the buffer, it is possible to prevent buffer overflow. In other words, since the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data to be held, it is possible to prevent an increase in the dynamic range of the buffer and therefore an increase in costs.

[0086] For example, the luminance component and color component (color difference component) of the coefficient data may be clipped using an upper limit value obtained by subtracting 1 from the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data during inverse adaptive color conversion.Furthermore, the luminance component and color component (color difference component) of the coefficient data may be clipped using a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1.

[0087] For example, the upper limit actResMax of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (34) using a value obtained by subtracting 1 from the dynamic range of the buffer. Also, the lower limit actResMin of the clipping process performed on coefficient data that has been adaptively color converted in a lossless manner is set as shown in the following equation (35) using a value obtained by subtracting 1 from the dynamic range of the buffer.

[0088] TIFF0007747236000034.tif12110...(34) TIFF0007747236000035.tif12108...(35)

[0089] That is, the upper limit value actResMax is set to a value obtained by subtracting 1 from the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer.The lower limit value actResMin is set to a value obtained by multiplying the exponent of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1.The upper limit value actResMax and the lower limit value actResMin are then used to clip the luminance component and chrominance component (chrominance component) of the coefficient data that has been adaptively color converted using a lossless method, as shown in the above equations (18) to (20).

[0090] Then, the coefficient data thus clipped is subjected to inverse YCgCo-R conversion as shown in the above equations (12) to (15), to derive coefficient data in the RGB domain.

[0091] By doing so, the hardware limitations of the buffer can be satisfied, and the occurrence of buffer overflow can be suppressed. Also, an increase in costs can be suppressed. Furthermore, since the value range of the coefficient data is limited by the clipping process, an increase in the load of the inverse adaptive color transform process can be suppressed. Furthermore, if the value range of the coefficient data is narrower than the dynamic range of the buffer, by doing so, an increase in distortion of the coefficient data after the inverse adaptive color transform can also be suppressed.

[0092] <Application example> Furthermore, the above-mentioned "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3" may be selectively applied. For example, the optimal method may be applied according to predetermined application conditions based on some information such as input coefficient data, hardware specifications, and load status. For example, when the dynamic range of the buffer is wider than the value range of the coefficient data, "Method 1-1" or "Method 2-1" may be applied. Furthermore, when the value range of the luminance component of the coefficient data and the value range of the color component (chrominance component) are the same, "Method 1," "Method 1-1," or "Method 1-2" may be applied.

[0093] In each method, either or both of the upper limit clipping and the lower limit clipping may be selectively applied. For example, if the lower limit clipping value is equal to or less than the lower limit of the coefficient data range, clipping of the lower limit value may be omitted (skip). Also, if the upper limit clipping value is equal to or greater than the upper limit of the coefficient data range, clipping of the upper limit value may be omitted (skip).

[0094] Of course, these are just examples, and the applicable conditions and the methods applied to each condition are not limited to these examples.

[0095] 2. First Embodiment <Inverse adaptive color conversion device> The present technology described above can be applied to any device. Fig. 2 is a block diagram showing an example of the configuration of an inverse adaptive color transformation device, which is one aspect of an image processing device to which the present technology is applied. The inverse adaptive color transformation device 100 shown in Fig. 2 is a device that performs a reversible inverse adaptive color transformation (inverse YCgCo-R transformation) on YCgCo domain coefficient data that has been subjected to a reversible adaptive color transformation (YCgCo-R transformation) on RGB domain coefficient data related to an image.

[0096] Note that Fig. 2 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 2. In other words, the inverse adaptive color transformation device 100 may have processing units that are not shown as blocks in Fig. 2, or may have processing or data flows that are not shown as arrows, etc. in Fig. 2.

[0097] As shown in FIG. 2, the inverse adaptive color transform device 100 includes a selector 101, a clip processor 102, and an inverse YCgCo-R converter 103.

[0098] The selection unit 101 acquires coefficient data res_x' to be input to the inverse adaptive color transformation device 100. The selection unit 101 also acquires cu_act_enabled_flag to be input to the inverse adaptive color transformation device 100. cu_act_enabled_flag is flag information indicating whether adaptive color transformation (inverse adaptive color transformation) is applicable. If this cu_act_enabled_flag is true (for example, "1"), it indicates that adaptive color transformation (inverse adaptive color transformation) is applicable. If this cu_act_enabled_flag is false (for example, "0"), it indicates that application of adaptive color transformation (inverse adaptive color transformation) is prohibited (i.e., cannot be applied).

[0099] The selection unit 101 selects whether to perform inverse adaptive color transformation on the coefficient data res_x' based on this cu_act_enabled_flag. For example, if cu_act_enabled_flag is true (e.g., "1"), the selection unit 101 determines that the coefficient data res_x' is YCgCo domain coefficient data obtained by YCgCo-R transforming coefficient data in the RGB domain. Then, the selection unit 101 supplies the coefficient data res_x' to the clip processing unit 102 so that the inverse YCgCo-R transform is performed on the coefficient data res_x'.

[0100] For example, when cu_act_enabled_flag is false (e.g., "0"), the selection unit 101 determines that the coefficient data res_x' is coefficient data in the RGB domain. Then, the selection unit 101 outputs the coefficient data res_x' to the outside of the inverse adaptive color transformation device 100 as coefficient data res_x after the inverse adaptive color transformation. In other words, the coefficient data in the RGB domain is output to the outside of the inverse adaptive color transformation device 100.

[0101] The clipping unit 102 acquires the coefficient data res_x' supplied from the selection unit 101. The clipping unit 102 also acquires variables such as log2MaxDR and BitDepth input to the inverse adaptive color transform device 100. log2MaxDR indicates the dynamic range of the buffer that stores the coefficient data res_x' during the inverse adaptive color transform process. BitDepth indicates the bit depth of the coefficient data res_x'.

[0102] The clipping unit 102 performs clipping on the coefficient data res_x' using upper and lower limit values ​​derived based on variables such as log2MaxDR and BitDepth. The clipping unit 102 supplies the coefficient data res_x' after the clipping to the inverse YCgCo-R conversion unit 103.

[0103] The inverse YCgCo-R converter 103 acquires the coefficient data res_x' that has been subjected to clipping, supplied from the clipping processor 102. The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the acquired coefficient data res_x' to generate inverse YCgCo-R converted coefficient data res_x. The inverse YCgCo-R converter 103 outputs the generated coefficient data res_x to the outside of the inverse adaptive color transform device 100. In other words, the coefficient data in the RGB domain is output to the outside of the inverse adaptive color transform device 100.

[0104] <Application of this technology to an inverse adaptive color conversion device> In such an inverse adaptive color transform device 100, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied.

[0105] For example, the clipping unit 102 clips the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R conversion unit 103 performs inverse adaptive color conversion on the coefficient data res_x' clipped at that level by the clipping unit 102 in a lossless manner.

[0106] In this way, the clipping unit 102 can clip the coefficient data res_x' at a level that does not increase distortion in the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion in the coefficient data res_x after the inverse adaptive color transform.

[0107] In addition, the inverse adaptive color conversion device 100 can apply the various methods of the present technology described above in <1. Clip processing of reversible inverse adaptive color conversion> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3").

[0108] For example, the clipping processing unit 102 may clip the luminance component and the color component (color difference component) of the coefficient data res_x' at the same level as described above in <Method 1>.

[0109] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' that has been clipped in this way, to derive the coefficient data res_x. In this way, the clipping processor 102 can perform clipping more easily and reduce an increase in load, compared to when clipping is performed at different levels for each component.

[0110] For example, as described above in <Method 1-1>, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x'. Alternatively, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x', by -1. Furthermore, the clipping unit 102 may perform both such clipping of the upper limit value and clipping of the lower limit value.

[0111] The inverse YCgCo-R converter 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' that has been clipped in this manner, deriving coefficient data res_x. By doing so, the clipping processor 102 can widen the range of values ​​between the upper and lower limit values ​​of the clipping process to be wider than the range of values ​​that the coefficient data res_x' can theoretically take. Therefore, the inverse adaptive color transform device 100 can suppress distortion of the coefficient data res_x after the inverse adaptive color transform. In other words, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while achieving lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain.

[0112] Also, for example, the clipping processing unit 102 may clip the coefficient data res_x' at a level based on its bit depth and the dynamic range of the buffer that holds the coefficient data res_x' during the inverse adaptive color transformation, as described above in <Method 1-2>.

[0113] Then, the inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' that has been clipped in this manner, to derive the coefficient data res_x. By clipping the coefficient data res_x' in this manner while taking into account the dynamic range (hardware limitations) of the buffer, the inverse adaptive color transform device 100 can prevent the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data res_x' to be stored, an increase in the dynamic range of the buffer can be prevented, and an increase in the cost of the inverse adaptive color transform device 100 can be prevented.

[0114] For example, the clipping unit 102 may perform clipping at a level derived using the smaller of a value based on the bit depth of the coefficient data res_x' and not based on the dynamic range of the buffer holding the coefficient data res_x', and a value not based on the bit depth and based on the dynamic range of the buffer.

[0115] Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this manner to derive the coefficient data res_x. By setting the upper and lower limits using the smaller value in this manner (i.e., by clipping the coefficient data res_x' to a narrower value range), the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and suppress an increase in the distortion of the coefficient data res_x after the inverse adaptive color transform, while suppressing an increase in the load of the inverse adaptive color transform process.

[0116] For example, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data res_x'. Alternatively, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data res_x', by -1. Furthermore, the clipping unit 102 may perform both such upper limit clipping and lower limit clipping.

[0117] Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this manner to derive the coefficient data res_x. In this manner, the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and suppress an increase in distortion of the coefficient data res_x after the inverse adaptive color transform, while suppressing an increase in the load of the inverse adaptive color transform process.

[0118] For example, the clipping unit 102 may clip the luminance component and the chrominance component of the coefficient data res_x' at their respective levels (first and second levels) as described in <Method 2>. This allows the clipping unit 102 to clip the coefficient data of each component at a value range appropriate to that component. The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' thus clipped, thereby deriving the coefficient data res_x. Therefore, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0119] For example, as described in <Method 2-1>, the difference between the upper and lower limit values ​​of the second level may be wider than the difference between the upper and lower limit values ​​of the first level. The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this manner, to derive the coefficient data res_x. By making the difference between the upper and lower limit values ​​of clipping for the coefficient data res_x' of the Cg and Co components, which are color difference components, wider than the difference between the upper and lower limit values ​​of clipping for the coefficient data res_x' of the Y component, which is a luminance component, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0120] For example, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2 where the bit depth of the coefficient data res_x' is the exponent. Alternatively, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using a lower limit value obtained by multiplying a power of 2 where the bit depth of the coefficient data res_x' is the exponent by -1. Furthermore, the clipping unit 102 may perform both such upper limit and lower limit clipping for the luminance component.

[0121] Alternatively, the clipping unit 102 may clip the color components (chrominance components) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x'. Furthermore, the clipping unit 102 may clip the color components (chrominance components) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by adding 1 to the bit depth of the coefficient data res_x', by -1. Furthermore, the clipping unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value for the color components (chrominance components). Furthermore, the clipping unit 102 may perform both clipping of the upper limit value and clipping of the lower limit value for each of the luminance component and the color components (chrominance components) as described above.

[0122] The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the luminance component and color component (color difference component) of the coefficient data res_x' that have been clipped in this manner, to derive the coefficient data res_x. In this way, the clipping processing unit 102 can clip the coefficient data res_x' of each component within a range corresponding to that component. Therefore, the inverse adaptive color transform device 100 can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color transform.

[0123] Also, for example, the clipping processing unit 102 may clip the coefficient data res_x' at a level (first level and second level) based on the bit depth and the dynamic range of the buffer that holds the coefficient data res_x' during the inverse adaptive color transformation, as described above in <Method 2-2>.

[0124] The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the luminance component and color component (color difference component) of the coefficient data res_x' that have been clipped in this manner, thereby deriving the coefficient data res_x. In this way, the inverse adaptive color transform device 100 can prevent buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data res_x' to be stored, an increase in the dynamic range of the buffer can be prevented, and an increase in the cost of the inverse adaptive color transform device 100 can be prevented.

[0125] For example, the clipping unit 102 may perform clipping on the luminance component of the coefficient data res_x' at a level derived using the smaller of a value based on the bit depth of the coefficient data res_x' but not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value based on the dynamic range of the buffer but not based on the bit depth.Also, the clipping unit 102 may perform clipping on the color component (chrominance component) of the coefficient data res_x' at a level derived using the smaller of a value obtained by adding 1 to the bit depth of the coefficient data res_x' but not based on the dynamic range of the buffer that holds the coefficient data res_x', and a value based on the dynamic range of the buffer but not based on the value obtained by adding 1 to the bit depth.

[0126] Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this way, to derive the coefficient data res_x. By setting the upper and lower limits using the smaller value in this way (i.e., by clipping the coefficient data to a narrower value range), the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color transform.

[0127] For example, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2 where the power is the smaller of the bit depth of the coefficient data res_x' or the dynamic range of the buffer that stores the coefficient data res_x'. Alternatively, the clipping unit 102 may clip the luminance component of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2 where the power is the smaller of the bit depth of the coefficient data res_x' or the dynamic range of the buffer that stores the coefficient data res_x' by 1. Furthermore, the clipping unit 102 may perform both such upper limit and lower limit clipping for the luminance component.

[0128] Alternatively, the clipping unit 102 may clip the color components (chrominance components) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from a power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data res_x'. Alternatively, the clipping unit 102 may clip the color components (chrominance components) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the power is the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' or the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data res_x', by -1. Alternatively, the clipping unit 102 may clip both the upper limit value and the lower limit value for the color components (chrominance components). Furthermore, the clipping processing unit 102 may perform both the upper limit clipping and the lower limit clipping for each of the luminance component and the color component (color difference component) as described above.

[0129] Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this manner, to derive the coefficient data res_x. In this manner, the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer, and can further suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color transform.

[0130] For example, the clipping unit 102 may clip the coefficient data res_x' at a level based on the dynamic range of the buffer that holds the coefficient data res_x' during inverse adaptive color transformation, as described above in <Method 3>.

[0131] The inverse YCgCo-R converter 103 performs inverse YCgCo-R conversion on the coefficient data res_x' that has been clipped in this manner, deriving the coefficient data res_x. In this way, the inverse adaptive color transform device 100 can prevent buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be stored, an increase in the dynamic range of the buffer can be prevented, and an increase in the cost of the inverse adaptive color transform device 100 can be prevented.

[0132] For example, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using an upper limit value obtained by subtracting 1 from the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer that stores the coefficient data res_x' during inverse adaptive color conversion. Alternatively, the clipping unit 102 may clip the luminance component and color component (color difference component) of the coefficient data res_x' using a lower limit value obtained by multiplying the power of 2, where the exponent is the value obtained by subtracting 1 from the dynamic range of the buffer, by -1.

[0133] The inverse YCgCo-R transform unit 103 then performs inverse YCgCo-R transform on the coefficient data res_x' that has been clipped in this manner, deriving the coefficient data res_x. By doing so, the inverse adaptive color transform device 100 can satisfy the hardware limitations of the buffer and prevent buffer overflow. Furthermore, an increase in the cost of the inverse adaptive color transform device 100 can be suppressed. Furthermore, since the range of coefficient data is limited by clipping, the inverse adaptive color transform device 100 can prevent an increase in the load of the inverse adaptive color transform process. Furthermore, by doing so, when the range of coefficient data is narrower than the dynamic range of the buffer, the inverse adaptive color transform device 100 can also prevent an increase in distortion of the coefficient data after the inverse adaptive color transform.

[0134] The inverse adaptive color transformation device 100 can be applied to the various application examples described above in <Application Examples>.

[0135] <Flow of inverse adaptive color conversion processing> Next, an example of the flow of the reverse adaptive color transformation process executed by the reverse adaptive color transformation device 100 will be described with reference to the flowchart of FIG.

[0136] When the inverse adaptive color transformation process is started, the selection unit 101 of the inverse adaptive color transformation device 100 determines in step S101 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S102.

[0137] In step S102, the clipping processor 102 clips the coefficient data res_x' using predetermined upper and lower limit values.

[0138] In step S103, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped in step S102, and derives coefficient data res_x after inverse adaptive color transformation.

[0139] In step S104, the inverse YCgCo-R converter 103 outputs the derived coefficient data res_x to the outside of the inverse adaptive color conversion device 100. When the process of step S104 ends, the inverse adaptive color conversion process ends.

[0140] Also, if it is determined in step S101 that cu_act_enabled_flag is false, the process proceeds to step S105.

[0141] In step S105, the selection unit 101 outputs the coefficient data res_x' as coefficient data res_x after the inverse adaptive color transformation to the outside of the inverse adaptive color transformation device 100. When the process of step S105 ends, the inverse adaptive color transformation process ends.

[0142] <Application of this technology to inverse adaptive color conversion processing> In such inverse adaptive color conversion processing, the present technology described above in <1. Clipping processing of lossless inverse adaptive color conversion> can be applied.

[0143] For example, in step S102, the clipping unit 102 clips the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, in step S103, the inverse YCgCo-R converter 103 performs reversible inverse adaptive color conversion on the coefficient data res_x' clipped at that level by the clipping unit 102.

[0144] In this way, the clipping unit 102 can clip the coefficient data res_x' at a level that does not increase distortion in the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform device 100 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion in the coefficient data res_x after the inverse adaptive color transform.

[0145] In this inverse adaptive color conversion process, the various methods of the present technology described above in <1. Clipping process of reversible inverse adaptive color conversion> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an inverse adaptive color conversion device>.

[0146] 3. Second Embodiment <Inverse quantization and inverse transformation device> The above-described inverse adaptive color transform device 100 (inverse adaptive color transform processing) can be applied to an inverse quantization and inverse transform device that performs inverse quantization and inverse transform processing. FIG. 4 is a block diagram showing an example of the configuration of an inverse quantization and inverse transform device, which is one aspect of an image processing device to which the present technology is applied. The inverse quantization and inverse transform device 200 shown in FIG. 4 is a device that performs lossless inverse adaptive color transform (YCgCo-R transform) on RGB domain coefficient data related to an image, orthogonal transforms the quantized coefficients, inverse quantizes them using a method corresponding to the quantization, inverse orthogonal transforms them using a method corresponding to the orthogonal transform, and performs lossless inverse adaptive color transform (inverse YCgCo-R transform). This processing (processing performed by the inverse quantization and inverse transform device 200) is also referred to as inverse quantization and inverse transform processing.

[0147] Note that Fig. 4 shows the main processing units, data flows, etc., and does not necessarily show everything. In other words, in the inverse quantization and inverse transform device 200, there may be processing units that are not shown as blocks in Fig. 4, and there may be processing and data flows that are not shown as arrows or the like in Fig. 4.

[0148] As shown in FIG. 4, the inverse quantization and inverse transform device 200 includes an inverse quantization unit 201 , an inverse orthogonal transform unit 202 , and an inverse adaptive color transform unit 203 .

[0149] The inverse quantization unit 201 acquires a quantization coefficient qcoef_x. This quantization coefficient qcoef_x is derived by quantizing the orthogonal transform coefficient coef_x using a predetermined method. The inverse quantization unit 201 also acquires parameters necessary for inverse quantization, such as a quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag. The transform_skip_flag is flag information indicating whether or not to skip (omit) the inverse orthogonal transform process. For example, if the transform_skip_flag is true, the inverse orthogonal transform process is skipped. If the transform_skip_flag is false, the inverse orthogonal transform process is executed.

[0150] The inverse quantization unit 201 uses these parameters to inverse quantize the quantized coefficient qcoef_x by a predetermined method corresponding to the above-mentioned quantization, and derives the orthogonal transform coefficient coef_x. The inverse quantization unit 201 supplies the derived orthogonal transform coefficient coef_x to the inverse orthogonal transform unit 202.

[0151] The inverse orthogonal transform unit 202 acquires the orthogonal transform coefficient coef_x supplied from the inverse quantization unit 201. This orthogonal transform coefficient coef_x is derived by orthogonally transforming the coefficient data res_x' using a predetermined method. The inverse orthogonal transform unit 202 also acquires parameters necessary for inverse quantization, such as transform information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx. mts_idx is an MTS (Multiple Transform Selection) identifier. lfnst_idx is mode information related to the low-frequency secondary transform. The inverse orthogonal transform unit 202 uses these parameters to perform an inverse orthogonal transform on the orthogonal transform coefficient coef_x using a predetermined method corresponding to the above-mentioned orthogonal transform, thereby deriving coefficient data res_x' that has been adaptively color transformed in a lossless manner. The inverse orthogonal transform unit 202 supplies the derived coefficient data res_x' to the inverse adaptive color transform unit 203.

[0152] The inverse adaptive color transform unit 203 acquires coefficient data res_x' supplied from the inverse orthogonal transform unit 202. This coefficient data res_x' is as described in the first embodiment. The inverse adaptive color transform unit 203 also acquires cu_act_enabled_flag. Based on cu_act_enabled_flag, the inverse adaptive color transform unit 203 appropriately performs inverse adaptive color transform (inverse YCgCo-R transform) on the coefficient data res_x' in a lossless manner, and derives coefficient data res_x after the inverse adaptive color transform process. This coefficient data res_x is as described in the first embodiment, and is coefficient data in the RGB domain.

[0153] The inverse adaptive color transform unit 203 outputs the derived coefficient data res_x to the outside of the inverse quantization and inverse transform device 200 .

[0154] <Application of this technology to an inverse quantization and inverse transform device> The present technology described above in <1. Clipping process of lossless inverse adaptive color transformation> can be applied to such an inverse quantization and inverse transformation device 200. That is, the inverse adaptive color transformation device 100 described in the first embodiment can be applied to the inverse adaptive color transformation unit 203 of the inverse quantization and inverse transformation device 200. In this case, the inverse adaptive color transformation unit 203 has the same configuration as the inverse adaptive color transformation device 100 and performs the same processing.

[0155] For example, in the inverse adaptive color transform unit 203, the clipping processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clipping processing unit 102, and derives coefficient data res_x after the inverse adaptive color transform processing. Then, the inverse YCgCo-R transform unit 103 outputs the derived coefficient data res_x to the outside of the inverse quantization and inverse transform device 200.

[0156] By doing so, the inverse adaptive color transform unit 203 can clip the coefficient data res_x' at a level that does not increase distortion in the coefficient data res_x after the inverse adaptive color transform. Therefore, similar to the inverse adaptive color transform device 100, the inverse adaptive color transform unit 203 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion in the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform device 200 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion in the coefficient data res_x to be output.

[0157] Then, as in the case of the inverse adaptive color transform device 100, the inverse adaptive color transform unit 203 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the inverse quantization and inverse transform device 200 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform>. By applying any of these methods, the inverse quantization and inverse transform device 200 can obtain the same effect as that described in <Application of the Present Technology to an Inverse Adaptive Color Transform Device>.

[0158] <Flow of inverse quantization and inverse transform processing> Next, an example of the flow of the inverse quantization and inverse transform process executed by the inverse quantization and inverse transform device 200 will be described with reference to the flowchart of FIG.

[0159] When the inverse quantization and inverse transform process starts, in step S201, the inverse quantization unit 201 of the inverse quantization and inverse transform device 200 inverse quantizes the quantized coefficients qcoef_x corresponding to each component identifier cIdx=0, 1, 2 included in the TU to be processed, and derives the orthogonal transform coefficients coef_x.

[0160] In step S202, the inverse orthogonal transform unit 202 refers to the transform information Tinfo corresponding to each component identifier, performs inverse orthogonal transform on the orthogonal transform coefficient coef_x, and derives coefficient data res_x'.

[0161] In step S203, the inverse adaptive color transform unit 203 executes inverse adaptive color transform processing, performs inverse YCgCo-R transform on the coefficient data res_x', and derives coefficient data res_x after the inverse adaptive color transform processing.

[0162] When the process of step S203 ends, the inverse quantization and inverse transform process ends.

[0163] <Application of this technology to inverse quantization and inverse transform processing> In such inverse quantization and inverse transform processing, the present technology described above in <1. Clip processing of lossless inverse adaptive color transform> can be applied. That is, as the inverse adaptive color transform processing of step S203, the inverse adaptive color transform processing described with reference to the flowchart in FIG. 3 can be applied.

[0164] In this way, the inverse adaptive color transform unit 203 can clip the coefficient data res_x' at a level that does not increase distortion in the coefficient data res_x after the inverse adaptive color transform. Therefore, the inverse adaptive color transform unit 203 can suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion in the coefficient data res_x after the inverse adaptive color transform.

[0165] That is, by performing the inverse quantization and inverse transform processing as described above, the inverse quantization and inverse transform device 200 can suppress an increase in the load of the inverse quantization and inverse transform processing while suppressing an increase in distortion of the coefficient data res_x to be output.

[0166] Note that in this inverse quantization inverse transform process (the inverse adaptive color transform process (step S203)), the various methods of the present technology described above in <1. Clipping process of inverse adaptive color transform using lossless method> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an inverse quantization inverse transform device> (i.e., the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0167] 4. Third Embodiment <Image decoding device> The above-described inverse quantization and inverse transform device 200 (inverse quantization and inverse transform processing) can be applied to an image decoding device. FIG. 6 is a block diagram showing an example of the configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 400 shown in FIG. 6 is a device that decodes coded data of a moving image. For example, the image decoding device 400 decodes coded data of a moving image coded using a coding method such as VVC, AVC, or HEVC described in the above-mentioned non-patent document. For example, the image decoding device 400 can decode coded data (bitstream) generated by an image coding device 500 (FIG. 8) described later.

[0168] Note that Fig. 6 shows the main processing units, data flows, etc., and is not limited to all that are shown in Fig. 6. In other words, in the image decoding device 400, there may be processing units that are not shown as blocks in Fig. 6, or there may be processing or data flows that are not shown as arrows, etc. in Fig. 6. This is also true for other figures that explain processing units, etc. within the image decoding device 400.

[0169] 6, the image decoding device 400 includes a control unit 401, an accumulation buffer 411, a decoding unit 412, an inverse quantization and inverse transform unit 413, a calculation unit 414, an in-loop filter unit 415, a rearrangement buffer 416, a frame memory 417, and a prediction unit 418. The prediction unit 418 includes an intra prediction unit and an inter prediction unit, both of which are not shown.

[0170] <Control unit> The control unit 401 executes processing related to decoding control. For example, the control unit 401 acquires coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, filter information Finfo, etc.) included in the bitstream via the decoding unit 412. The control unit 401 may also estimate coding parameters not included in the bitstream. Furthermore, the control unit 401 controls each processing unit (accumulation buffer 411 to prediction unit 418) of the image decoding device 400 based on the acquired (or estimated) coding parameters, thereby controlling decoding.

[0171] For example, the control unit 401 supplies header information Hinfo to the inverse quantization and inverse transform unit 413, the prediction unit 418, and the in-loop filter unit 415. Furthermore, the control unit 401 supplies prediction mode information Pinfo to the inverse quantization and inverse transform unit 413 and the prediction unit 418. Furthermore, the control unit 401 supplies transformation information Tinfo to the inverse quantization and inverse transform unit 413. Furthermore, the control unit 401 supplies residual information Rinfo to the decoding unit 412. Furthermore, the control unit 401 supplies filter information Finfo to the in-loop filter unit 415.

[0172] Of course, the above example is merely an example and is not limiting. For example, each encoding parameter may be supplied to any processing unit. Furthermore, other information may be supplied to any processing unit.

[0173] <Header information Hinfo> The header information Hinfo may include information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header (PH), and a slice header (SH).

[0174] For example, the header information Hinfo may include information specifying an image size (e.g., parameters such as PicWidth indicating the horizontal width of an image and PicHeight indicating the vertical width of an image). The header information Hinfo may also include information specifying a bit depth (e.g., parameters such as bitDepthY indicating the bit depth of a luminance component and bitDepthC indicating the bit depth of a chrominance component). The header information Hinfo may also include a parameter ChromaArrayType indicating a chrominance array type. The header information Hinfo may also include parameters such as MaxCUSize indicating the maximum value of a CU size and MinCUSize indicating the minimum value of a CU size. The header information Hinfo may also include parameters such as MaxQTDepth indicating the maximum depth of quad-tree partitioning and MinQTDepth indicating the minimum depth. The header information Hinfo may also include parameters such as MaxBTDepth indicating the maximum depth of binary-tree partitioning and MinBTDepth indicating the minimum depth. Furthermore, the header information Hinfo may include a parameter such as MaxTSSize indicating the maximum value of the transform skip block (also referred to as the maximum transform skip block size).

[0175] The header information Hinfo may also include information specifying an on / off flag (also called a validity flag) for each encoding tool. For example, the header information Hinfo may include an on / off flag related to an orthogonal transform process or a quantization process. The on / off flag for an encoding tool may also be interpreted as a flag indicating whether or not syntax related to that encoding tool is present in the encoded data. Alternatively, a value of 1 (true) of the on / off flag may indicate that the encoding tool is available, and a value of 0 (false) may indicate that the encoding tool is unavailable. The interpretation of the flag value (true or false) may be reversed.

[0176] <Prediction mode information Pinfo> The prediction mode information Pinfo may include, for example, a parameter such as PBSize indicating the size (prediction block size) of the PB to be processed (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and other information.

[0177] The intra prediction mode information IPinfo may include, for example, prev_intra_luma_pred_flag, mpm_idx, rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and information such as IntraPredModeY indicating the luma intra prediction mode derived from that syntax.

[0178] Furthermore, the intra prediction mode information IPinfo may include, for example, ccp_flag (cclmp_flag), mclm_flag, chroma_sample_loc_type_idx, chroma_mpm_idx, and information such as IntraPredModeC indicating a luma intra prediction mode derived from these syntaxes.

[0179] ccp_flag (cclmp_flag) is an inter-component prediction flag, and is flag information indicating whether inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0180] "mclm_flag" is a multi-class linear prediction mode flag, which is information relating to the linear prediction mode (linear prediction mode information). More specifically, "mclm_flag" is flag information indicating whether to use the multi-class linear prediction mode. For example, "mclm_flag==0" indicates a one-class mode (single-class mode) (e.g., CCLMP), and "mclm_flag==1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0181] chroma_sample_loc_type_idx is a chrominance sample location type identifier, which is an identifier that identifies the type of pixel location of the chrominance component (also referred to as the chrominance sample location type). Note that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted as chroma_sample_loc_info() (i.e., stored in chroma_sample_loc_info()). This chroma_sample_loc_info() is information about the pixel location of the chrominance component.

[0182] chroma_mpm_idx is a chrominance MPM identifier, and is an identifier indicating which prediction mode candidate in the chrominance intra prediction mode candidate list (intraPredModeCandListC) is to be specified as the chrominance intra prediction mode.

[0183] The motion prediction information MVinfo may include information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0184] Of course, any information may be included in the prediction mode information Pinfo, and information other than the above information may be included in the prediction mode information Pinfo.

[0185] <Conversion information Tinfo> The transformation information Tinfo may include information such as TBWSize, TBHSize, ts_flag, scanIdx, a quantization parameter qP, and a quantization matrix scaling_matrix (see, for example, JCTVC-W1005, 7.3.4 Scaling list data syntax).

[0186] TBWSize is a parameter indicating the width size of the transform block to be processed. Note that the transform information Tinfo may include the logarithm of TBWSize with base 2, log2TBWSize, instead of this TBWSize. TBHSize is a parameter indicating the height size of the transform block to be processed. Note that the transform information Tinfo may include the logarithm of TBHSize with base 2, log2TBHSize, instead of this TBHSize.

[0187] ts_flag is a transform skip flag, which is flag information indicating whether to skip the (inverse) primary transform and the (inverse) secondary transform. scanIdx is a scan identifier.

[0188] Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included in the conversion information Tinfo.

[0189] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax of JCTVC-W1005) may include, for example, the following information:

[0190] cbf(coded_block_flag): residual data presence flag last_sig_coeff_x_pos: Last non-zero coefficient X coordinate last_sig_coeff_y_pos: Last non-zero coefficient Y coordinate coded_sub_block_flag: Sub-block non-zero coefficient presence flag sig_coeff_flag: Non-zero coefficient presence flag gr1_flag: Flag indicating whether the level of the non-zero coefficient is greater than 1 (also called the GR1 flag) gr2_flag: Flag indicating whether the level of the non-zero coefficient is greater than 2 (also called the GR2 flag) sign_flag: Sign indicating whether the non-zero coefficient is positive or negative (also called the sign code) coeff_abs_level_remaining: Non-zero coefficient residual level (also called non-zero coefficient residual level)

[0191] Of course, the residual information Rinfo may include any information, and may include information other than the above information.

[0192] <Filter information Finfo> The filter information Finfo may include, for example, control information regarding each of the following filter processes:

[0193] Control information for the deblocking filter (DBF) Control information for pixel adaptive offset (SAO) Control information for the adaptive loop filter (ALF) Other control information for linear and nonlinear filters

[0194] Further, for example, the information may include information specifying the picture to which each filter is applied, an area within the picture, on / off control information for a filter in units of CU, on / off control information for a filter relating to a boundary between slices and tiles, etc. Of course, any information may be included in the filter information Finfo, and information other than the above information may be included.

[0195] <Accumulation buffer> The accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400. At a predetermined timing, or when a predetermined condition is met, the accumulation buffer 411 extracts coded data included in the accumulated bitstream and supplies the coded data to the decoding unit 412.

[0196] <Decryption section> The decoding unit 412 acquires the coded data supplied from the accumulation buffer 411, and entropy decodes (losslessly decodes) the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, thereby deriving coding parameters.

[0197] The coding parameters may include, for example, header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, filter information Finfo, etc. That is, the decoding unit 412 decodes and parses (analyzes and obtains) this information from the bitstream.

[0198] The decoding unit 412 executes such processing (decoding, parsing, etc.) under the control of the control unit 401, and supplies the obtained information to the control unit 401.

[0199] Furthermore, the decoding unit 412 decodes the coded data by referring to the residual information Rinfo. At this time, the decoding unit 412 applies entropy decoding (lossless decoding) such as CABAC or CAVLC. That is, the decoding unit 412 decodes the coded data using a decoding method corresponding to the coding method of the coding process executed by the coding unit 514 of the image coding device 500.

[0200] For example, assume that CABAC is applied. The decoding unit 412 performs arithmetic decoding on the encoded data using a context model, and derives the quantization coefficient level of each coefficient position in each transform block. The decoding unit 412 supplies the derived quantization coefficient level to the inverse quantization and inverse transform unit 413.

[0201] <Inverse quantization and inverse transform section> The inverse quantization and inverse transform unit 413 obtains the quantization coefficient level supplied from the decoding unit 412. The inverse quantization and inverse transform unit 413 obtains coding parameters such as prediction mode information Pinfo and transform information Tinfo supplied from the control unit 401.

[0202] The inverse quantization and inverse transform unit 413 performs inverse quantization and inverse transform processing on the quantized coefficients level based on coding parameters such as prediction mode information Pinfo and transform information Tinfo, thereby deriving residual data D'. This inverse quantization and inverse transform processing is the inverse processing of the transform quantization processing performed by the transform quantization unit 513 (FIG. 8), which will be described later. That is, in the inverse quantization and inverse transform processing, for example, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are performed. Inverse quantization is the inverse processing of the quantization performed by the transform quantization unit 513. Inverse orthogonal transform is the inverse processing of the orthogonal transform performed by the transform quantization unit 513. Inverse adaptive color transform is the inverse processing of the adaptive color transform performed by the transform quantization unit 513. Of course, any processing may be included in the inverse quantization and inverse transform processing, and some of the above-described processing may be omitted, or processing other than the above-described processing may be included. The inverse quantization and inverse transform unit 413 supplies the derived residual data D' to the calculation unit 414.

[0203] <Arithmetic section> The calculation unit 414 obtains the residual data D' supplied from the inverse quantization and inverse transform unit 413 and the predicted image supplied from the prediction unit 418. The calculation unit 414 adds the residual data D' and the predicted image P corresponding to the residual data D' to derive a locally decoded image. The calculation unit 414 supplies the derived locally decoded image to the in-loop filter unit 415 and the frame memory 417.

[0204] <In-loop filter section> The in-loop filter unit 415 acquires the locally decoded image supplied from the calculation unit 414. The in-loop filter unit 415 acquires filter information Finfo supplied from the control unit 401. Note that any information may be input to the in-loop filter unit 415, and information other than the above information may also be input.

[0205] The in-loop filter unit 415 performs appropriate filtering on the locally decoded image based on the filter information Finfo. For example, the in-loop filter unit 415 may apply a bilateral filter as the filtering process. For example, the in-loop filter unit 415 may apply a deblocking filter (DBF (DeBlocking Filter)) as the filtering process. For example, the in-loop filter unit 415 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as the filtering process. For example, the in-loop filter unit 415 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as the filtering process. Furthermore, the in-loop filter unit 415 may apply a combination of multiple of these filters as the filtering process. Note that which filters to apply and in what order they are applied are arbitrary and can be selected as appropriate. For example, the in-loop filter unit 415 applies four in-loop filters, namely a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filtering process.

[0206] The in-loop filter unit 415 performs filtering corresponding to the filtering performed by the encoding device. For example, the in-loop filter unit 415 performs filtering corresponding to the filtering performed by an in-loop filter unit 518 (FIG. 8) of the image encoding device 500, which will be described later. Of course, the filtering performed by the in-loop filter unit 415 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 415 may apply a Wiener filter or the like.

[0207] The in-loop filter unit 415 supplies the filtered locally decoded image to a reordering buffer 416 and a frame memory 417 .

[0208] <Sorting buffer> The reordering buffer 416 receives the locally decoded images supplied from the in-loop filter unit 415 as input and holds (stores) them. The reordering buffer 416 uses the locally decoded images to reconstruct decoded images for each picture and holds them (stores them in the buffer). The reordering buffer 416 reorders the obtained decoded images from decoding order to playback order. The reordering buffer 416 outputs the group of decoded images reordered in playback order to the outside of the image decoding device 400 as video data.

[0209] <Frame memory> The frame memory 417 acquires the locally decoded image supplied from the calculation unit 414, reconstructs the decoded image for each picture, and stores the reconstructed image in a buffer within the frame memory 417. The frame memory 417 also acquires the locally decoded image that has been in-loop filtered and supplied from the in-loop filter unit 415, reconstructs the decoded image for each picture, and stores the reconstructed image in a buffer within the frame memory 417.

[0210] The frame memory 417 appropriately supplies the stored decoded image (or a part thereof) as a reference image to the prediction unit 418. Note that the frame memory 417 may also store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like related to generation of the decoded image.

[0211] <Prediction Department> The prediction unit 418 obtains prediction mode information Pinfo supplied from the control unit 401. The prediction unit 418 also obtains a decoded image (or a part thereof) read from the frame memory 417. The prediction unit 418 performs prediction processing in the prediction mode adopted during encoding based on the prediction mode information Pinfo, and generates a predicted image P by referring to the decoded image as a reference image. The prediction unit 418 supplies the generated predicted image P to the calculation unit 414.

[0212] <Application of this technology to image decoding devices> The present technology described above in <1. Clipping process of lossless inverse adaptive color transform> may be applied to such an image decoding device 400. That is, the image decoding device 400 may apply the inverse quantization and inverse transform device 200 described in the second embodiment as the inverse quantization and inverse transform unit 413. In this case, the inverse quantization and inverse transform unit 413 has the same configuration as the inverse quantization and inverse transform device 200 ( FIG. 4 ) and performs the same process.

[0213] For example, in the inverse quantization and inverse transform unit 413, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the decoding unit 412 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 inverse quantizes the quantization coefficient qcoef_x using information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag, to derive the orthogonal transform coefficient coef_x.

[0214] The inverse orthogonal transform unit 202 performs inverse orthogonal transform on the orthogonal transform coefficients coef_x derived by the inverse quantization unit 201 using information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx, to derive coefficient data res_x'.

[0215] Based on cu_act_enabled_flag, the inverse adaptive color transform unit 203 performs inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner as appropriate on the coefficient data res_x' derived by the inverse orthogonal transform unit 202, to derive coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.

[0216] As described in the first embodiment, the inverse adaptive color transform unit 203 has the same configuration and performs the same processing as the inverse adaptive color transform device 100. For example, in the inverse adaptive color transform unit 203, the clipping processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clipping processing unit 102, to derive coefficient data res_x after the inverse adaptive color transform processing. Then, the inverse YCgCo-R transform unit 103 supplies the derived coefficient data res_x to the calculation unit 414 as residual data D'.

[0217] In this way, the inverse quantization and inverse transform unit 413 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x after inverse adaptive color transform. Therefore, similar to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 413 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the output coefficient data res_x (residual data D'). In other words, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0218] Then, as in the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 413 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the image decoding device 400 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform>. By applying any of these methods, the image decoding device 400 can obtain the same effect as that described in <Application of the Present Technology to an Inverse Quantization and Inverse Transform Device> (that is, the same effect as that described in <Application of the Present Technology to an Inverse Adaptive Color Transform Device>).

[0219] <Flow of image decoding process> Next, a description will be given of the flow of each process executed by the above-described image decoding device 400. First, an example of the flow of image decoding process will be described with reference to the flowchart in FIG.

[0220] When the image decoding process starts, in step S401, the accumulation buffer 411 acquires and holds (accumulates) a bitstream (encoded data) supplied from outside the image decoding device 400.

[0221] In step S402, the decoding unit 412 executes a decoding process. For example, the decoding unit 412 parses (analyzes and acquires) various coding parameters (for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, etc.) from the bitstream. The control unit 401 supplies the acquired various coding parameters to various processing units, thereby setting the various coding parameters.

[0222] Furthermore, the control unit 401 sets the unit of processing based on the obtained coding parameters. Furthermore, the decoding unit 412 decodes the bitstream under the control of the control unit 401, and derives the quantization coefficient level.

[0223] In step S403, the inverse quantization and inverse transform unit 413 performs inverse quantization and inverse transform processing to derive residual data D'.

[0224] In step S404, the prediction unit 418 generates a predicted image P. For example, the prediction unit 418 performs a prediction process using a prediction method specified by the encoding side based on the encoding parameters set in step S402, and generates the predicted image P by referring to a reference image stored in the frame memory 417, for example.

[0225] In step S405, the calculation unit 414 adds the residual data D' obtained in step S403 and the predicted image P obtained in step S404 to derive a locally decoded image.

[0226] In step S406, the in-loop filter unit 415 performs in-loop filtering on the locally decoded image obtained by the processing in step S405.

[0227] In step S407, the reordering buffer 416 derives decoded images using the locally decoded images filtered in step S406, and reorders the order of the decoded images from the decoding order to the playback order. The decoded images reordered in the playback order are output to the outside of the image decoding device 400 as moving images.

[0228] In step S408, the frame memory 417 stores at least one of the locally decoded image obtained by the process in step S405 and the locally decoded image that has been filtered by the process in step S406.

[0229] When the process of step S408 ends, the image decoding process ends.

[0230] <Application of this technology to image decoding processing> In such an image decoding process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. That is, the inverse quantization and inverse transform process described with reference to the flowchart in Fig. 5 can be applied as the inverse quantization and inverse transform process of step S403.

[0231] In this way, the inverse quantization and inverse transform unit 413 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform unit 413 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform.

[0232] That is, by performing the image decoding process as described above, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0233] Note that in this image decoding process (the inverse quantization and inverse transform process (step S403)), the various methods of the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an image decoding device> (i.e., the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0234] 5. Fourth Embodiment <Image encoding device> The above-described inverse quantization and inverse transform device 200 (inverse quantization and inverse transform processing) can be applied to an image encoding device. FIG. 8 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 500 shown in FIG. 8 is a device that encodes image data of a moving image. For example, the image encoding device 500 encodes image data of a moving image using an encoding method such as VVC (Versatile Video Coding), AVC (Advanced Video Coding), or HEVC (High Efficiency Video Coding) described in the above-mentioned non-patent document. For example, the image encoding device 500 can generate encoded data (bitstream) that can be decoded by the above-mentioned image decoding device 400 (FIG. 6).

[0235] Note that Fig. 8 shows the main processing units, data flows, etc., and is not necessarily all that is shown in Fig. 8. In other words, in the image encoding device 500, there may be processing units that are not shown as blocks in Fig. 8, and there may be processing and data flows that are not shown as arrows, etc. in Fig. 8. This is also true for other figures that explain processing units, etc. within the image encoding device 500.

[0236] 8, the image coding device 500 includes a control unit 501, a rearrangement buffer 511, a calculation unit 512, a transform / quantization unit 513, a coding unit 514, and an accumulation buffer 515. The image coding device 500 also includes a dequantization / inverse transform unit 516, a calculation unit 517, an in-loop filter unit 518, a frame memory 519, a prediction unit 520, and a rate control unit 521.

[0237] <Control unit> The control unit 501 divides the video data held in the rearrangement buffer 511 into blocks (CU, PU, ​​TU, etc.) of processing units based on an externally or pre-specified block size of processing units. The control unit 501 also determines coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization). For example, the control unit 501 can set a transform skip flag, etc.

[0238] After determining the above-described coding parameters, the control unit 501 supplies them to each block. For example, the header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the coding unit 514 and the prediction unit 520. The transformation information Tinfo is supplied to the coding unit 514, the transform / quantization unit 513, and the inverse quantization / inverse transformation unit 516. The filter information Finfo is supplied to the coding unit 514 and the in-loop filter unit 518. Of course, the destination to which each coding parameter is supplied is arbitrary and is not limited to this example.

[0239] <Sorting buffer> Each field (input image) of video data is input to the image coding device 500 in its playback order (display order). The reordering buffer 511 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 501, the reordering buffer 511 reorders the input images in coding order (decoding order) and divides them into blocks, which are processing units. The reordering buffer 511 supplies each processed input image to the calculation unit 512.

[0240] <Arithmetic section> The calculation unit 512 subtracts the predicted image P supplied from the prediction unit 520 from the image corresponding to the block of processing units supplied from the sorting buffer 511 to derive residual data D, and supplies it to the transformation and quantization unit 513.

[0241] <Transformation and Quantization Unit> The transform and quantization unit 513 acquires the residual data D supplied from the calculation unit 512. The transform and quantization unit 513 also acquires the prediction mode information Pinfo and the transformation information Tinfo supplied from the control unit 501. The transform and quantization unit 513 performs a transform and quantization process on the residual data D based on the prediction mode information Pinfo and the transformation information Tinfo, and derives a quantization coefficient level. The transform and quantization process may include, for example, adaptive color transform, orthogonal transform, and quantization. Of course, any process may be included in the transform and quantization process, and some of the processes described above may be omitted, or processes other than those described above may be included. The transform and quantization unit 513 supplies the derived quantization coefficient level to the encoding unit 514 and the inverse quantization and inverse transform unit 516.

[0242] <Encoding section> The encoding unit 514 acquires the quantization coefficient level supplied from the transform / quantization unit 513. The encoding unit 514 also acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 501. The encoding unit 514 also acquires information related to filters, such as filter coefficients, supplied from the in-loop filter unit 518. The encoding unit 514 also acquires information related to an optimal prediction mode supplied from the prediction unit 520.

[0243] The encoding unit 514 performs entropy encoding (lossless encoding) on ​​the quantized coefficient level to generate a bit string (encoded data). The encoding unit 514 may apply, for example, CABAC (Context-based Adaptive Binary Arithmetic Code) as the entropy encoding. The encoding unit 514 may apply, for example, CAVLC (Context-based Adaptive Variable Length Code) as the entropy encoding. Of course, the content of this entropy encoding is arbitrary and is not limited to these examples.

[0244] Furthermore, the encoding unit 514 derives residual information Rinfo from the quantized coefficient levels, encodes the residual information Rinfo, and generates a bit string.

[0245] Furthermore, the encoding unit 514 includes information about the filter supplied from the in-loop filter unit 518 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 520 in the prediction mode information Pinfo. Then, the encoding unit 514 encodes the various encoding parameters described above (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) to generate a bit string.

[0246] The encoding unit 514 multiplexes the bit strings of the various types of information generated as described above to generate encoded data, and supplies the encoded data to the accumulation buffer 515.

[0247] <Accumulation buffer> The accumulation buffer 515 temporarily stores the coded data obtained by the coding unit 514. The accumulation buffer 515 outputs the stored coded data, for example, as a bit stream or the like, to the outside of the image coding device 500 at a predetermined timing. For example, this coded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, or the like. In other words, the accumulation buffer 515 also functions as a transmission unit that transmits the coded data (bit stream).

[0248] <Inverse quantization and inverse transform section> The inverse quantization and inverse transform unit 516 obtains the quantization coefficient level supplied from the transform and quantization unit 513. The inverse quantization and inverse transform unit 516 also obtains the transformation information Tinfo supplied from the control unit 501.

[0249] The inverse quantization and inverse transform unit 516 performs inverse quantization and inverse transform processing on the quantized coefficients level based on the transform information Tinfo, and derives residual data D'. This inverse quantization and inverse transform processing is the inverse processing of the transform and quantization processing performed in the transform and quantization unit 513, and is the same processing as the inverse quantization and inverse transform processing performed in the inverse quantization and inverse transform unit 413 of the image decoding device 400 described above.

[0250] That is, in the inverse quantization and inverse transform processing, for example, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are performed. This inverse quantization is the inverse process of the quantization performed in the transform quantization unit 513, and is the same process as the inverse quantization performed in the inverse quantization and inverse transform unit 413. Furthermore, the inverse orthogonal transform is the inverse process of the orthogonal transform performed in the transform quantization unit 513, and is the same process as the inverse orthogonal transform performed in the transform quantization unit 513. Furthermore, the inverse adaptive color transform is the inverse process of the adaptive color transform performed in the transform quantization unit 513, and is the same process as the inverse adaptive color transform performed in the transform quantization unit 513.

[0251] Of course, any process may be included in this inverse quantization and inverse transform process, and some of the processes described above may be omitted, or processes other than those described above may be included. The inverse quantization and inverse transform unit 516 supplies the derived residual data D' to the calculation unit 517.

[0252] <Arithmetic section> The calculation unit 517 obtains the residual data D' supplied from the inverse quantization and inverse transform unit 516 and the predicted image P supplied from the prediction unit 520. The calculation unit 517 adds the residual data D' to the predicted image P corresponding to the residual data D' to derive a locally decoded image. The calculation unit 517 supplies the derived locally decoded image to the in-loop filter unit 518 and the frame memory 519.

[0253] <In-loop filter section> The in-loop filter unit 518 acquires the locally decoded image supplied from the calculation unit 517. The in-loop filter unit 518 also acquires filter information Finfo supplied from the control unit 501. The in-loop filter unit 518 also acquires the input image (original image) supplied from the rearrangement buffer 511. Note that any information may be input to the in-loop filter unit 518, and information other than the above information may also be input. For example, information such as a prediction mode, motion information, a code amount target value, a quantization parameter qP, a picture type, and a block (CU, CTU, etc.) may be input to the in-loop filter unit 518 as necessary.

[0254] The in-loop filter unit 518 performs appropriate filtering on the locally decoded image based on the filter information Finfo. The in-loop filter unit 518 also uses the input image (original image) and other input information for the filtering, as necessary.

[0255] For example, the in-loop filter unit 518 may apply a bilateral filter as its filtering process. For example, the in-loop filter unit 518 may apply a deblocking filter (DBF (DeBlocking Filter)) as its filtering process. For example, the in-loop filter unit 518 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as its filtering process. For example, the in-loop filter unit 518 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as its filtering process. Furthermore, the in-loop filter unit 518 may apply a combination of multiple of these filters as its filtering process. Note that which filters to apply and in what order to apply them are arbitrary and can be selected as appropriate. For example, the in-loop filter unit 518 applies four in-loop filters, namely, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as its filtering process.

[0256] Of course, the filtering process performed by the in-loop filter unit 518 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 518 may apply a Wiener filter or the like.

[0257] The in-loop filter unit 518 supplies the filtered locally decoded image to the frame memory 519. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 518 supplies the information about the filter to the encoding unit 514.

[0258] <Frame memory> The frame memory 519 performs processing related to the storage of image-related data. For example, the frame memory 519 acquires and holds (stores) a locally decoded image supplied from the calculation unit 517 or a filtered locally decoded image supplied from the in-loop filter unit 518. The frame memory 519 also reconstructs and holds a decoded image for each picture using the locally decoded image (storing it in a buffer within the frame memory 519). The frame memory 519 supplies the decoded image (or a part thereof) to the prediction unit 520 in response to a request from the prediction unit 520.

[0259] <Prediction Department> The prediction unit 520 executes processing related to generation of a predicted image. For example, the prediction unit 520 acquires prediction mode information Pinfo supplied from the control unit 501. For example, the prediction unit 520 acquires an input image (original image) supplied from the reordering buffer 511. For example, the prediction unit 520 acquires a decoded image (or a part thereof) read from the frame memory 519.

[0260] The prediction unit 520 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and an input image (original image). That is, the prediction unit 520 performs prediction and motion compensation by referring to the decoded image as a reference image, and generates a predicted image P. The prediction unit 520 supplies the generated predicted image P to the calculation units 512 and 517. Furthermore, the prediction unit 520 supplies information on the prediction mode selected by the above processing, i.e., the optimal prediction mode, to the encoding unit 514 as necessary.

[0261] <Rate control section> The rate control unit 521 executes processing related to rate control. For example, the rate control unit 521 controls the rate of the quantization operation of the transform and quantization unit 513 based on the code amount of the coded data accumulated in the accumulation buffer 515 so as to prevent overflow or underflow.

[0262] <Transformation and Quantization Unit> Fig. 9 is a block diagram showing an example of the main configuration of the transform and quantization unit 513 in Fig. 8. As shown in Fig. 9, the transform and quantization unit 513 has an adaptive color transform unit 541, an orthogonal transform unit 542, and a quantization unit 543.

[0263] The adaptive color transform unit 541 acquires the residual data D ( FIG. 8 ) supplied from the calculation unit 512 as coefficient data res_x. The adaptive color transform unit 541 also acquires cu_act_enabled_flag supplied from the control unit 501. The adaptive color transform unit 541 performs adaptive color transform on the coefficient data res_x based on the value of cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (e.g., “1”), the adaptive color transform unit 541 performs lossless adaptive color transform (YCgCo-R transform) to convert the coefficient data res_x in the RGB domain into coefficient data res_x′ in the YCgCo domain. This adaptive color transform is the inverse process of the inverse adaptive color transform executed in the inverse quantization inverse transform unit 413 or the inverse quantization inverse transform unit 516. The adaptive color transform unit 541 supplies the derived coefficient data res_x′ to the orthogonal transform unit 542.

[0264] The orthogonal transform unit 542 acquires the coefficient data res_x' supplied from the adaptive color transform unit 541. The orthogonal transform unit 542 acquires information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501. The orthogonal transform unit 542 performs an orthogonal transform on the coefficient data res_x' using the acquired information, and derives the orthogonal transform coefficient coef_x. This orthogonal transform is the inverse process of the inverse orthogonal transform executed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The orthogonal transform unit 542 supplies the derived orthogonal transform coefficient coef_x to the quantization unit 543.

[0265] The quantization unit 543 acquires the orthogonal transform coefficient coeff_x supplied from the orthogonal transform unit 542. The quantization unit 543 also acquires information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501. The quantization unit 543 quantizes the orthogonal transform coefficient coef_x using the acquired information to derive the quantized coefficient qcoef_x. This quantization is the inverse process of the inverse quantization performed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The quantization unit 543 supplies the derived quantized coefficient qcoef_x to the encoding unit 514 and the inverse quantization and inverse transform unit 516 (FIG. 8) as the quantized coefficient level.

[0266] <Adaptive color conversion section> Fig. 10 is a block diagram showing an example of the main configuration of adaptive color conversion section 541 in Fig. 9. As shown in Fig. 10, adaptive color conversion section 541 has a selection section 571 and a YCgCo-R conversion section 572.

[0267] The selection unit 571 acquires the residual data D ( FIG. 8 ) supplied from the calculation unit 512 as coefficient data res_x. The selection unit 571 also acquires cu_act_enabled_flag supplied from the control unit 501. The selection unit 571 selects whether or not to perform adaptive color conversion on the acquired coefficient data res_x based on the value of cu_act_enabled_flag. For example, if cu_act_enabled_flag is true (e.g., "1"), the selection unit 571 determines that adaptive color conversion can be applied, and supplies the coefficient data res_x to the YCgCo-R conversion unit 572.

[0268] For example, if cu_act_enabled_flag is false (e.g., "0"), the selection unit 571 determines that the application of adaptive color transformation (inverse adaptive color transformation) is prohibited (i.e., cannot be applied), and supplies the coefficient data res_x to the orthogonal transformation unit 542 (Figure 9) as coefficient data res_x' after the applied color transformation.

[0269] The YCgCo-R transform unit 572 acquires the coefficient data res_x supplied from the selection unit 101. The YCgCo-R transform unit 572 performs YCgCo-R transform on the acquired coefficient data res_x, and derives YCgCo-R transformed coefficient data res_x'. This YCgCo-R transform is the inverse process of the inverse YCgCo-R transform executed in the inverse quantization and inverse transform unit 413 and the inverse quantization and inverse transform unit 516. The YCgCo-R transform unit 572 supplies the derived coefficient data res_x' to the orthogonal transform unit 542 (FIG. 9).

[0270] <Application of this technology to image encoding devices> In such an image coding device 500 (FIG. 8), the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. That is, the image coding device 500 can apply the inverse quantization and inverse transform device 200 described in the second embodiment as the inverse quantization and inverse transform unit 516. In this case, the inverse quantization and inverse transform unit 516 has the same configuration as the inverse quantization and inverse transform device 200 (FIG. 4) and performs the same process.

[0271] For example, in the inverse quantization and inverse transform unit 516, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the transform and quantization unit 513 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 uses information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501 to inverse quantize the quantization coefficient qcoef_x, and derive the orthogonal transform coefficient coef_x.

[0272] The inverse orthogonal transform unit 202 performs inverse orthogonal transform on the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201 using information such as transformation information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501, and derives coefficient data res_x'.

[0273] The inverse adaptive color transform unit 203 performs inverse adaptive color transform (inverse YCgCo-R transform) in a lossless manner as appropriate on the coefficient data res_x' derived by the inverse orthogonal transform unit 202 based on cu_act_enabled_flag supplied from the control unit 501, to derive coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.

[0274] As described in the first embodiment, the inverse adaptive color transform unit 203 has the same configuration and performs the same processing as the inverse adaptive color transform device 100. For example, in the inverse adaptive color transform unit 203, the clipping processing unit 102 clips the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped at that level by the clipping processing unit 102, to derive coefficient data res_x after the inverse adaptive color transform processing. Then, the inverse YCgCo-R transform unit 103 supplies the derived coefficient data res_x to the calculation unit 517 as residual data D'.

[0275] In this way, the inverse quantization and inverse transform unit 516 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x after inverse adaptive color transform. Therefore, similar to the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 516 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the output coefficient data res_x (residual data D'). In other words, the image encoding device 500 can suppress an increase in the load of the image encoding process while suppressing a decrease in the quality of the decoded image.

[0276] Then, as in the case of the inverse quantization and inverse transform device 200, the inverse quantization and inverse transform unit 516 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the image encoding device 500 may apply the various methods of the present technology described above in <1. Clipping Process of Lossless Inverse Adaptive Color Transform>. By applying any of these methods, the image encoding device 500 can obtain the same effect as that described in <Application of the Present Technology to an Inverse Quantization and Inverse Transform Device> (that is, the same effect as that described in <Application of the Present Technology to an Inverse Adaptive Color Transform Device>).

[0277] <Image encoding process flow> Next, an example of the flow of the image encoding process executed by the image encoding device 500 described above will be described with reference to the flowchart of FIG.

[0278] When the image encoding process starts, in step S501, the reordering buffer 511 is controlled by the control unit 501 to reorder the frames of the input video data from the display order to the encoding order.

[0279] In step S502, the control unit 501 sets processing units for the input image held in the sorting buffer 511 (divides the image into blocks).

[0280] In step S503, the control unit 501 sets coding parameters (for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, etc.) for the input image held by the sorting buffer 511.

[0281] In step S504, the prediction unit 520 performs a prediction process to generate a predicted image etc. in an optimal prediction mode. For example, in this prediction process, the prediction unit 520 performs intra prediction to generate a predicted image etc. in an optimal intra prediction mode, performs inter prediction to generate a predicted image etc. in an optimal inter prediction mode, and selects an optimal prediction mode from among them based on a cost function value etc.

[0282] In step S505, the calculation unit 512 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S504. That is, the calculation unit 512 derives residual data D between the input image and the predicted image. The residual data D derived in this way has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to when the image is encoded as is.

[0283] In step S506, the transform / quantization unit 513 performs a transform / quantization process on the residual data D derived by the process of step S505 using encoding parameters such as the transform information Tinfo set in step S503, and derives the quantization coefficient level.

[0284] In step S507, the inverse quantization and inverse transform unit 516 performs inverse quantization and inverse transform processing on the quantization coefficient level derived in step S506 using coding parameters such as the transform information Tinfo set in step S503, to derive residual data D'. This inverse quantization and inverse transform processing is the inverse processing of the transform quantization processing in step S506, and is similar to the inverse quantization and inverse transform processing in step S403 of the image decoding processing in Figure 7.

[0285] In step S508, the calculation unit 517 generates a locally decoded decoded image by adding the predicted image generated by the prediction process in step S504 to the residual data D' derived by the inverse quantization and inverse transform process in step S507.

[0286] In step S509, the in-loop filter unit 518 performs in-loop filtering on the locally decoded image derived by the processing in step S508.

[0287] In step S510, the frame memory 519 stores the locally decoded image derived by the process in step S508 and the locally decoded image filtered in step S509.

[0288] In step S511, the encoding unit 514 encodes the quantization coefficient level derived by the transform and quantization process in step S506 to derive encoded data. At this time, the encoding unit 514 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo). Furthermore, the encoding unit 514 derives residual information RInfo from the quantization coefficient level and encodes the residual information RInfo.

[0289] In step S512, the accumulation buffer 515 accumulates the encoded data derived in this manner and outputs it to the outside of the image encoding device 500, for example, as a bit stream. This bit stream is transmitted to a decoding side device (for example, the image decoding device 400) via, for example, a transmission path or a recording medium. In addition, the rate control unit 521 controls the rate as necessary. When the process of step S512 ends, the image encoding process ends.

[0290] <Flow of transformation and quantization process> Next, an example of the flow of the transform and quantization process executed in step S506 in FIG. 11 will be described with reference to the flowchart in FIG.

[0291] When the conversion and quantization process is started, in step S541, the adaptive color conversion unit 541 adaptively color converts the coefficient data res_x derived by the processing of step S505 based on the cu_act_enabled_flag set in step S503 (Figure 11), and derives coefficient data res_x'.

[0292] In step S542, the orthogonal transform unit 542 performs an orthogonal transform on the coefficient data res_x' derived in step S541 using the transform information Tinfo, prediction mode information Pinfo, etc. set in step S503 (FIG. 11), to derive an orthogonal transform coefficient coef_x.

[0293] In step S543, the quantization unit 543 quantizes the orthogonal transform coefficient coef_x derived in step S542 using the transformation information Tinfo etc. set in step S503 (FIG. 11) to derive the quantized coefficient qcoef_x.

[0294] When the process of step S543 ends, the process returns to FIG.

[0295] <Adaptive color conversion process flow> Next, an example of the flow of the adaptive color conversion process executed in step S541 of FIG. 12 will be described with reference to the flowchart of FIG.

[0296] When the adaptive color conversion process starts, the selection unit 571 of the adaptive color conversion unit 541 determines in step S571 whether or not cu_act_enabled_flag is true. If it is determined that cu_act_enabled_flag is true, the process proceeds to step S572.

[0297] In step S572, the YCgCo-R conversion unit 572 performs YCgCo-R conversion on the coefficient data res_x of the three components to derive coefficient data res_x' after adaptive color conversion.

[0298] In step S573, the YCgCo-R conversion unit 572 supplies the derived coefficient data res_x' to the orthogonal transformation unit 542. When the process of step S573 ends, the adaptive color conversion process ends.

[0299] Also, if it is determined in step S571 that cu_act_enabled_flag is false, the process proceeds to step S574.

[0300] In step S574, the selection unit 571 supplies the coefficient data res_x as coefficient data res_x' after adaptive color conversion to the orthogonal transformation unit 542. When the process of step S574 ends, the adaptive color conversion process ends.

[0301] <Application of this technology to image encoding processing> In such an image encoding process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. That is, the inverse quantization and inverse transform process described with reference to the flowchart in Fig. 5 can be applied as the inverse quantization and inverse transform process of step S507.

[0302] In this way, the inverse quantization and inverse transform unit 516 can clip the coefficient data res_x' at a level that does not increase distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform. Therefore, the inverse quantization and inverse transform unit 516 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transform.

[0303] That is, by performing the image coding process as described above, the image coding device 500 can suppress an increase in the load of the image coding process while suppressing a decrease in the quality of the decoded image.

[0304] Note that in this image encoding process (the inverse quantization and inverse transform process (step S507)), the various methods of the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3") can be applied. By applying any of these methods, it is possible to obtain the same effect as that described in <Application of the present technology to an image encoding device> (i.e., the same effect as that described in <Application of the present technology to an inverse adaptive color transform device>).

[0305] <6. Notes> <Computer> The above-described series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed on a computer. Here, the term "computer" includes computers built into dedicated hardware, and general-purpose personal computers, etc., that can execute various functions by installing various programs.

[0306] FIG. 14 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0307] In a computer 800 shown in FIG. 14, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0308] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0309] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0310] In the computer configured as above, the CPU 801 executes the above-described series of processes by, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data necessary for the CPU 801 to execute various processes as appropriate.

[0311] The program executed by the computer can be applied by recording it on removable media 821 such as package media, for example. In this case, the program can be installed in storage unit 813 via input / output interface 810 by inserting removable media 821 into drive 815.

[0312] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0313] Alternatively, this program can be installed in advance in the ROM 802 or the storage unit 813 .

[0314] <Applicable targets of this technology> This technology can be applied to any image encoding or decoding method. In other words, as long as it does not conflict with the above-mentioned technology, the specifications of various processes related to image encoding and decoding, such as transform (inverse transform), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the above-mentioned examples. Furthermore, as long as it does not conflict with the above-mentioned technology, some of these processes may be omitted.

[0315] The present technology can also be applied to a multi-viewpoint image coding system that codes a multi-viewpoint image including images from a plurality of views. The present technology can also be applied to a multi-viewpoint image decoding system that decodes coded data of a multi-viewpoint image including images from a plurality of views. In that case, the present technology can be applied in the coding and decoding of each view.

[0316] Furthermore, the present technology can be applied to a hierarchical image coding (scalable coding) system that codes hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. Also, the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes coded data of hierarchical images that are layered (hierarchized) so as to have a scalability function for a predetermined parameter. In this case, the present technology can be applied in the coding and decoding of each layer.

[0317] Furthermore, although the above describes the inverse adaptive color transformation device 100, the inverse quantization and inverse transformation device 200, the image decoding device 400, and the image encoding device 500 as application examples of the present technology, the present technology can be applied to any configuration.

[0318] For example, this technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) used in satellite broadcasting, cable TV and other wired broadcasting, distribution over the Internet, and distribution to terminals via cellular communications, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, or play images from these storage media.

[0319] Furthermore, for example, the present technology can also be implemented as a part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0320] Furthermore, for example, the present technology can also be applied to a network system configured with multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and collaborate on processing via a network. For example, the present technology may be implemented in a cloud service that provides image (video)-related services to any terminal, such as a computer, AV (Audio Visual) equipment, a portable information processing terminal, or an IoT (Internet of Things) device.

[0321] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0322] <Fields and uses where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications thereof are also arbitrary.

[0323] For example, the present technology can be applied to systems and devices used to provide viewing content, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for transportation, such as monitoring traffic conditions and controlling automatic driving. Furthermore, for example, the present technology can also be applied to systems and devices used for security. Furthermore, for example, the present technology can also be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can also be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can also be applied to systems and devices used to monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can also be applied to systems and devices used for sports.

[0324] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Therefore, the value that this "flag" can take may be, for example, two values, 1 / 0, or three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, identification information (including flags) can be assumed not only to include the identification information in the bit stream, but also to include difference information of the identification information relative to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information relative to the reference information.

[0325] Furthermore, various types of information (metadata, etc.) related to the coded data (bitstream) may be transmitted or recorded in any form as long as they are associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. In other words, data associated with each other may be combined into one piece of data or may be individual pieces of data. For example, information associated with coded data (image) may be transmitted over a transmission path separate from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium separate from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may refer to only a portion of the data, rather than the entire data. For example, an image and information corresponding to that image may be associated with each other in any unit, such as multiple frames, one frame, or a portion of a frame.

[0326] In this specification, terms such as "composite," "multiplex," "add," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0327] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present technology.

[0328] For example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0329] Furthermore, for example, the above-described program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0330] Also, for example, each step of a single flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0331] Furthermore, the program executed by the computer may have the following features. For example, the processing of the steps of writing the program may be executed in chronological order according to the order described in this specification. The processing of the steps of writing the program may also be executed in parallel. Furthermore, the processing of the steps of writing the program may be executed individually at the necessary timing, such as when called. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of writing the program may be executed in parallel with the processing of another program. Furthermore, the processing of the steps of writing the program may be executed in combination with the processing of another program.

[0332] Furthermore, for example, multiple technologies related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any multiple technologies can also be implemented in combination. For example, part or all of the present technology described in any embodiment can be implemented in combination with part or all of the present technology described in another embodiment. Furthermore, part or all of any of the above-described present technologies can be implemented in combination with other technologies not described above.

[0333] The present technology can also be configured as follows. (1) a clipping processing unit that clips coefficient data that has been adaptively color transformed in a lossless manner at a level based on the bit depth of the coefficient data; an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped at the level by the clip processing unit in the lossless manner; An image processing device comprising: (2) The clipping processing unit clips the luminance component and the color component of the coefficient data at the same level. The image processing device according to (1). (3) The clip processing unit clipping the luminance component and the color component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, where the power is the value obtained by adding 1 to the bit depth; The luminance component and the color component of the coefficient data are clipped using a lower limit value obtained by multiplying a power of 2, where the value obtained by adding 1 to the bit depth is used as an exponent, by -1. (2) An image processing device according to the present invention. (4) The level is a value based on the bit depth and the dynamic range of a buffer that stores the coefficient data. (2) An image processing device according to the present invention. (5) The level is a value derived using the smaller of a value based on the bit depth and not based on the dynamic range of the buffer, and a value not based on the bit depth and based on the dynamic range of the buffer. (4) An image processing device according to (4). (6) The clip processing unit clipping the luminance component and the color component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer; The luminance component and the color component of the coefficient data are clipped to a lower limit value obtained by multiplying the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer by −1. (5) An image processing device according to (5). (7) The clip processing unit clipping a luminance component of the coefficient data at a first level; Clipping the color components of the coefficient data at a second level The image processing device according to (1). (8) The second level has a wider difference between the upper limit and the lower limit than the first level. (7) An image processing device according to (7). (9) The clip processing unit clipping the luminance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2 where the bit depth is the exponent; clipping the luminance component of the coefficient data to a lower limit value obtained by multiplying a power of 2, with the bit depth as an exponent, by −1; clipping the color components of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, where the power is the value obtained by adding 1 to the bit depth; The color components of the coefficient data are clipped using a lower limit value obtained by multiplying a power of 2, where the value obtained by adding 1 to the bit depth is used as an exponent, by −1. (8) An image processing device according to (8). (10) The first level and the second level are values ​​based on the bit depth and a dynamic range of a buffer that stores the coefficient data. (7) An image processing device according to (7). (11) The clip processing unit clipping the luminance component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of the bit depth and the dynamic range of the buffer; clipping the luminance component of the coefficient data to a lower limit value obtained by multiplying a power of 2, which is the smaller of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, by −1; clipping the color components of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, the power being the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer; The color components of the coefficient data are clipped to a lower limit value obtained by multiplying the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer by −1. (10) An image processing device according to (10). (12) The level is a value based on the dynamic range of a buffer that stores the coefficient data. The image processing device according to (1). (13) The clip processing unit clipping the luminance component and the color component of the coefficient data to an upper limit value obtained by subtracting 1 from a power of 2, where the exponent is a value obtained by subtracting 1 from the dynamic range of the buffer; The luminance component and the color component of the coefficient data are clipped to a lower limit value obtained by multiplying a power of 2 obtained by subtracting 1 from the dynamic range of the buffer and multiplying the power by -1. (12) An image processing device according to (12). (14) An inverse orthogonal transform unit that performs inverse orthogonal transform on orthogonal transform coefficients to generate the coefficient data that has been adaptively color transformed in the lossless manner, The clipping unit is configured to clip the coefficient data generated by the inverse orthogonal transform unit at the level. The image processing device according to (1). (15) The image processing apparatus further includes an inverse quantization unit that inversely quantizes the quantized coefficients and generates the orthogonal transform coefficients, The inverse orthogonal transform unit is configured to perform an inverse orthogonal transform on the orthogonal transform coefficients generated by the inverse quantization unit. (14) An image processing device according to (14). (16) A decoding unit that decodes encoded data and generates the quantized coefficients, The inverse quantization unit is configured to inverse quantize the quantized coefficients generated by the decoding unit. (15) An image processing device according to (15). (17) The inverse adaptive color transform unit is configured to perform inverse adaptive color transform on the coefficient data clipped at the level by the clip processing unit in the lossless manner to generate prediction residual data of image data, a calculation unit that adds prediction data of the image data to the prediction residual data generated by the inverse adaptive color transform unit to generate the image data; (16) An image processing device according to (16). (18) A transform and quantization unit that performs adaptive color transform on image data in a lossless manner to generate the coefficient data, orthogonally transforms the coefficient data to generate the orthogonal transform coefficients, and quantizes the orthogonal transform coefficients to generate the quantized coefficients; a coding unit that generates coded data by coding the quantized coefficients generated by the transform and quantization unit; Furthermore, The inverse quantization unit is configured to inverse quantize the quantized coefficients generated by the transform and quantization unit. (15) An image processing device according to (15). (19) A calculation unit that subtracts prediction data of the image data from the image data to generate prediction residual data, The transform and quantization unit is configured to perform adaptive color transform on the prediction residual data generated by the calculation unit in a lossless manner to generate the coefficient data, orthogonally transform the coefficient data to generate the orthogonal transform coefficients, and quantize the orthogonal transform coefficients to generate the quantized coefficients. (18) An image processing device according to (18). (20) Clipping the coefficient data that has been adaptively color transformed in a lossless manner at a level based on the bit depth of the coefficient data; The coefficient data clipped at the level is inversely adaptively color transformed in the lossless manner. Image processing methods. [Explanation of symbols]

[0334] 100 inverse adaptive color transform device, 101 selection unit, 102 clip processing unit, 103 inverse YCgCo-R transform unit, 200 inverse quantization and inverse transform device, 201 inverse quantization unit, 202 inverse orthogonal transform unit, 203 inverse adaptive color transform unit, 400 image decoding device, 401 control unit, 412 decoding unit, 413 inverse quantization and inverse transform unit, 414 calculation unit, 415 in-loop filter unit, 416 sorting buffer, 417 frame memory, 418 prediction unit, 500 image encoding device, 501 control unit, 511 sorting buffer, 512 calculation unit, 313 transform and quantization unit, 514 encoding unit, 515 accumulation buffer, 516 inverse quantization and inverse transform unit, 517 calculation unit, 518 In-loop filter unit, 519 frame memory, 520 prediction unit, 521 rate control unit

Claims

1. a clipping processing unit that clips coefficient data that has been adaptively color converted using a lossless method, with an upper limit value being a value obtained by subtracting 1 from a power of 2, where the power value is the value obtained by adding 1 to the bit depth of the coefficient data, and a lower limit value being a value obtained by multiplying the power of 2, where the power value is the value obtained by adding 1 to the bit depth, by −1; an inverse adaptive color transform unit that performs inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value by the clip processing unit in the lossless manner; An image processing device comprising:

2. The clipping processing unit clips the luminance component and the color component of the coefficient data at the same upper limit value and lower limit value. The image processing device according to claim 1 .

3. the inverse adaptive color transform unit performs inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value by the clipping processing unit in the lossless manner to generate prediction residual data of the image data; a calculation unit that generates the image data by adding prediction data of the image data to the prediction residual data generated by the inverse adaptive color transform unit; The image processing device according to claim 1 .

4. Clipping the coefficient data adaptively color converted in a lossless manner to an upper limit value obtained by subtracting 1 from a power of 2, where the power is the value obtained by adding 1 to the bit depth of the coefficient data, and a lower limit value obtained by multiplying the power of 2, where the power is the value obtained by adding 1 to the bit depth, by -1; performing an inverse adaptive color transform on the coefficient data clipped at the upper limit value and the lower limit value in the lossless manner; An image processing method comprising:

Images