Image processing device and method

Abstract

The present disclosure relates to an image processing device and method that makes it possible to suppress increases in the distortion of coefficient data after inverse adaptive color transformation while also suppressing increases in the load of the inverse adaptive color transformation process. The coefficient data obtained through reversible adaptive color transformation is clipped at a level based on the bit depth of the coefficient data, and the coefficient data clipped at this level is subjected to reversible inverse adaptive color transformation. The present disclosure is applicable to, for example, image processing devices, image encoding devices, image decoding devices, transmitting devices, receiving devices, transmitting / receiving devices, information processing devices, image capture devices, playback devices, electronic devices, image processing methods, information processing methods, and the like.

Description

Technical Field

[0001] The disclosure relates to an image processing apparatus and method, and more particularly, to an image processing apparatus and method capable of suppressing an increase in the load of an inverse adaptive color transform process while suppressing an increase in distortion of coefficient data after inverse adaptive color transform. Background Art

[0002] Conventionally, a coding method for deriving a prediction residual of a moving image, performing coefficient transformation, quantization, and encoding has been proposed (for example, see Non-Patent Document 1 and Non-Patent Document 2). In addition, as a coding tool for improving coding efficiency in RGB 444, Adaptive Color Transform (ACT) for transforming coefficient data in the RGB domain into coefficient data in the YCgCo domain has been proposed. In addition, a lossless method has been proposed as an adaptive color transform (for example, see Non-Patent Document 3).

[0003] At the same time, in lossy inverse adaptive color transform (inverse ACT), in order to suppress the increase in the load of inverse adaptive color transform processing, it has been proposed to use [-2^bitDepth, 2^bitDepth-1] to clip the coefficient data in the YCgCo domain used as the input signal (for example, see non-patent document 4).

[0004] Reference List

[0005] Non-patent literature

[0006] Non-Patent Document 1: Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, “Versatile Video Coding (Draft 7)”, JVET-P2001-vE, 16th Meeting of the Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC1 / SC 29 / WG 11: Geneva, Switzerland, October 1-11, 2019

[0007] Non-Patent Document 2: Jianle Chen, Yan Ye, Seung Hwan Kim, “Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7)”, JVET-P2002-v1, ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, Joint Video Experts Group (JVET) 16th Meeting: Geneva, Switzerland, October 1-11, 2019

[0008] Non-Patent Literature 3: Xiaoyu Xiu, Yi-Wen Chen, Tsung-Chuan Ma, Hong-Jheng Jhu, Xianglin Wang, Jie Zhao, Hendry, Seethal Paluri, Seung Hwan Kim, Weijia Zhu, Jizheng Xu, Li Zhang, “ACT color conversion for both lossless and lossy coding,” JVET-Q0510_r2, Joint Video Experts Group (JVET) of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 17th Meeting: Brussels, Belgium, January 1-17, 2020

[0009] Non-Patent Literature 4: Xiaoyu Xiu, Yi-Wen Chen, Tsung-Chuan Ma, Hong-Jheng Jhu, Xianglin Wang “AHG16: Clipping residual samples for ACT,” JVET-Q0513_r2, ITU-T SG16WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 Joint Video Experts Group (JVET) 17th Meeting: Brussels, Belgium, January 1-17, 2020 Summary of the Invention

[0010] Problems to be solved by the present invention

[0011] However, the value range of the coefficient data in the YCgCo domain that has been subjected to lossless adaptive color conversion is wider than the value range of the coefficient data in the YCgCo domain that has been subjected to adaptive color conversion using a lossy method. Therefore, when the method described in Non-Patent Document 4 is applied to the lossless inverse adaptive color conversion disclosed in Non-Patent Document 3, the coefficient data in the YCgCo domain is changed by clipping processing, and there is a possibility that the coefficient data in the RGB domain after the inverse adaptive color conversion may be distorted.

[0012] The disclosure has been made in view of such circumstances, and an object of the present disclosure is to suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in distortion of coefficient data after the inverse adaptive color transform.

[0013] Solution to the problem

[0014] According to one aspect of the present technology, an image processing device includes: a cropping processing unit configured to crop coefficient data subjected to lossless adaptive color transform at a level based on the bit depth of the coefficient data; and an inverse adaptive color transform unit configured to perform inverse adaptive color transform on the coefficient data cropped at the level by the cropping processing unit by a lossless method.

[0015] According to one aspect of the present technology, an image processing method includes: clipping coefficient data subjected to lossless adaptive color transform at a level based on the bit depth of the coefficient data; and performing inverse adaptive color transform on the coefficient data clipped at the level by a lossless method.

[0016] According to an aspect of the present invention, in an image processing apparatus and method, coefficient data subjected to lossless adaptive color transform is clipped at a level based on its bit depth, and the coefficient data clipped at the level is subjected to lossless inverse adaptive color transform. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a diagram for describing an example of the difference in value range between YCgCo conversion and YCgCo-R conversion.

[0018] Figure 2 is a block diagram showing a main configuration example of an inverse adaptive color conversion device.

[0019] Figure 3 is a flowchart illustrating an example of the flow of inverse adaptive color conversion processing.

[0020] Figure 4 It is a block diagram showing a main configuration example of an inverse quantization and inverse transform device.

[0021] Figure 5 : is a flowchart showing an example of the flow of inverse quantization and inverse transform processing.

[0022] Figure 6 is a block diagram showing a main configuration example of an image decoding device.

[0023] Figure 7 is a flowchart illustrating an example of the flow of image decoding processing.

[0024] Figure 8 It is a block diagram showing a main configuration example of an image encoding device.

[0025] Figure 9 : is a block diagram showing a main configuration example of a transform and quantization unit.

[0026] Figure 10 is a block diagram showing a main configuration example of an adaptive color conversion unit.

[0027] Figure 11 is a flowchart showing an example of the flow of image encoding processing.

[0028] Figure 12 is a flowchart for describing an example of the flow of transform and quantization processing.

[0029] Figure 13 is a flowchart for describing an example of the flow of adaptive color conversion processing.

[0030] Figure 14 is a block diagram showing a main configuration example of a computer. DETAILED DESCRIPTION

[0031] Hereinafter, modes for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. In addition, the embodiments will be described in the following order.

[0032] 1. Cropping process for lossless inverse adaptive color transformation

[0033] 2. First embodiment (inverse adaptive color conversion device)

[0034] 3. Second embodiment (inverse quantization and inverse conversion device)

[0035] 4. Third Embodiment (Image Decoding Device)

[0036] 5. Fourth embodiment (image encoding device)

[0037] 6. Appendix

[0038] <1. Cropping for lossless inverse adaptive color transformation>

[0039] <Documents supporting technical content or technical terminology, etc.>

[0040] The scope disclosed in the present technology includes not only the contents described in the embodiments but also the contents described in the following non-patent documents and the like known at the time of filing or the contents of other documents cited in the following non-patent documents and the like.

[0041] Non-patent document 1: (mentioned above)

[0042] Non-patent document 2: (mentioned above)

[0043] Non-Patent Document 3: (mentioned above)

[0044] Non-Patent Document 4: (mentioned above)

[0045] Non-Patent Literature 5: Recommendation ITU-T H.264 (04 / 2017) “Advanced video coding for generic audiovisual services”, April 2017

[0046] Non-Patent Document 6: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0047] In other words, the content described in the above non-patent literature is also the basis for determining the support conditions. For example, even if the quadtree block structure and the quadtree plus binary tree (QTBT) block structure described in the above non-patent literature are not directly described in the examples, the quadtree block structure and the quadtree plus binary tree (QTBT) block structure are still within the scope of the disclosure of the present technology and meet the support conditions of the claims. In addition, similarly, even if technical terms such as parsing, grammar, and semantics are not directly described in the examples, they are still within the scope of the disclosure of the present technology and meet the support conditions of the scope of the claims.

[0048] In addition, in this specification, unless otherwise specified, a "block" used as a partial region or processing unit of an image (picture) (not a block indicating a processing unit) indicates any partial region in a picture, and the size, shape, characteristics, etc. of the block are not limited. For example, a "block" includes any partial region (processing unit) described in the above-mentioned non-patent literature, such as a transform block (TB), a transform unit (TU), a prediction block (PB), a prediction unit (PU), a minimum coding unit (SCU), a coding unit (CU), a maximum coding unit (LCU), a coding tree block (CTB), a coding tree unit (CTU), a subblock, a macroblock, a tile, or a slice.

[0049] In addition, when specifying the size of such a block, the block size can be specified not only directly but also indirectly. For example, the block size can be specified using size information for identifying the size. In addition, for example, the block size can be specified by a ratio or difference with the size of a reference block (e.g., LCU, SCU, etc.). For example, when sending information for specifying the block size as a syntax element, etc., information for indirectly specifying the size as described above can be used as the information. In this way, the amount of information can be reduced and the coding efficiency can be improved. In addition, the specification of the block size also includes the specification of the range of the block size (e.g., the specification of the range of allowable block sizes, etc.).

[0050] In addition, in this specification, encoding includes not only all processes for converting an image into a bit stream, but also a part of the processes. For example, encoding includes not only processes including prediction processing, orthogonal transform, quantization, arithmetic coding, etc., but also processes collectively referred to as quantization and arithmetic coding, including prediction processing, quantization, and arithmetic coding. Similarly, decoding includes not only all processes for converting a bit stream into an image, but also a part of the processes. For example, decoding includes not only processes including inverse arithmetic decoding, inverse quantization, inverse orthogonal transform, prediction processing, etc., but also processes including inverse arithmetic decoding and inverse quantization, including inverse arithmetic decoding, inverse quantization, and prediction processing.

[0051] <Adaptive Color Transform>

[0052] As a coding tool for improving coding efficiency in RGB 444, an adaptive color transform (ACT) for transforming coefficient data in the RGB domain into coefficient data in the YCgCo domain has been proposed. Adaptive color transform is also called YCgCo transform. Note that the inverse process of adaptive color transform is also called inverse adaptive color transform (inverse ACT). The inverse process of YCgCo transform is also called inverse YCgCo transform. YCgCo transform (inverse YCgCo transform) is a lossy method. Therefore, in the YCgCo transform (inverse YCgCo transform), the transformation between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain may be lossy. In other words, by transforming the coefficient data in the RGB domain into YCgCo and then performing an inverse YCgCo transform, there is a possibility that it is difficult to generate (restore) the coefficient data in the RGB domain that is the same as the coefficient data in the RGB domain before the YCgCo transform.

[0053] On the other hand, in non-patent document 3, a lossless method has been proposed as an adaptive color transform. Lossless adaptive color transform is also called YCgCo-R transform. The inverse process of YCgCo-R transform is also called inverse YCgCo-R transform. That is, YCgCo-R transform (inverse YCgCo-R transform) is a lossless method. Therefore, in YCgCo-R transform (inverse YCgCo-R transform), lossless transformation between coefficient data in the RGB domain and coefficient data in the YCgCo domain can be achieved. In other words, by transforming the coefficient data in the RGB domain into YCgCo-R and then performing an inverse YCgCo-R transform, there is a possibility of generating (restoring) coefficient data in the RGB domain that is the same as the coefficient data in the RGB domain before the YCgCo-R transform.

[0054] Therefore, in the encoding of applying adaptive color transformation and inverse adaptive color transformation, when applying lossy YCgCo transformation and inverse YCgCo transformation, there is a possibility that lossless encoding is difficult to achieve. On the other hand, lossless encoding can be achieved by applying YCgCo-R transformation and inverse YCgCo-R transformation.

[0055] <YCgCo-R transformation and inverse YCgCo-R transformation>

[0056] The YCgCo-R transformation converts RGB components into YCgCo components as shown in the following equations (1) to (4).

[0057] Co = R - B

[0058] …(1)

[0059] t = B + (Co >> 1)

[0060] …(2)

[0061] Cg = G - t

[0062] …(3)

[0063] Y = t + (Cg >> 1)

[0064] …(4)

[0065] Through the YCgCo-R transformation as shown in equations (1) to (4), similar to the case of YCgCo transformation, the coefficient data in the RGB domain can be converted into the coefficient data in the YCgCo domain corresponding to the YCbCr domain only through simple shift operations and addition / subtraction. The inverse YCgCo-R transformation is performed as shown in the following equations (5) to (8).

[0066] t = Y - (Cg >> 1)

[0067] …(5)

[0068] G = Cg + t

[0069] …(6)

[0070] B = t - (Co >> 1)

[0071] …(7)

[0072] R = Co + B

[0073] …(8)

[0074] <Cropping process>

[0075] Meanwhile, Non-Patent Document 4 proposes clipping the coefficient data in the YCgCo domain, serving as the input signal, with [-2^bitDepth, 2^bitDepth-1] to suppress an increase in the load of the inverse adaptive color transform process in lossy inverse adaptive color transform (inverse ACT). [A, B] represents a value range, where A is the lower bound and B is the upper bound. Furthermore, "clipped with [A, B]" indicates that values ​​equal to or less than the lower bound A of the input signal are A, and values ​​equal to or greater than the upper bound B are B. bitDepth represents the bit depth of the coefficient data in the RGB domain before adaptive color transform is performed.

[0076] For example, the range of coefficient data at bit depth bitDepth is [-2^bitDepth, 2^bitDepth-1]. When YCgCo conversion is performed on coefficient data in the RGB domain within this range, the range of coefficient data for each component in the YCgCo domain is theoretically within the range of [-2^bitDepth, 2^bitDepth-1]. However, in practice, due to the influence of external factors, the coefficient data for each component in the YCgCo domain may take values ​​outside this range. In the inverse adaptive color conversion process, when the range of the input signal is expanded as described above, the load of the inverse adaptive color conversion process may increase. In particular, there is the possibility of increased costs when installing hardware.

[0077] Therefore, the coefficient data of each component in the YCgCo domain as the input signal of the inverse YCgCo transform is clipped by [-2^bitDepth, 2^bitDepth-1]. That is, the coefficient data r as the input signal of the inverse YCgCo transform Y [x][y]、r Cb [x][y] and r Cr [x][y] is processed as in the following equations (9) to (11).

[0078] r Y [x][y]=Clip3(-(1<<BitDepth), (1<<BitDepth)-1, r Y [x][y])

[0079] …(9)

[0080] r Cb [x][y]=Clip3(-(1<<BitDepth), (1<<BitDepth)-1, r Cb [x][y])

[0081] …(10)

[0082] r Cr[x][y]=Clip3(-(1<<BitDepth), (1<<BitDepth)-1, r Cr [x][y])

[0083] …(11)

[0084] r Y [x][y] represents the coefficient data of the Y component. Cb [x][y] represents the coefficient data of the Cg component. rCr[x][y] represents the coefficient data of the CO component. Clip3(A, B, C) represents a clipping function that clips C using the lower bound A and the upper bound B. << represents a bit shift (i.e., a power of 2).

[0085] In this way, inverse YCgCo conversion does not need to consider unnecessary processing at levels outside this value range. Therefore, it is possible to suppress increases in the load of inverse adaptive color conversion processing. In particular, when implementing hardware, it is possible to suppress increases in cost.

[0086] It is assumed that the coefficient data clipped in this manner is inversely YCgCo-transformed by the lossless method described in Non-Patent Document 3. In this case, the coefficient data rY[x][y], r Cb [x][y] and r Cr [x][y] is processed as in the following equations (12) to (15).

[0087] tmp=r Y [x][y]-(r Cb [x][y]>>1)

[0088] …(12)

[0089] r Y [x][y]=tmp+r Cb [x][y]

[0090] …(13)

[0091] r Cb [x][y]=tmp-(r Cr [x][y]>>1)

[0092] …(14)

[0093] r Cr [x][y]+=r Cb [x][y]

[0094] …(15)

[0095] Note that Equations (12) to (15) are equivalent to Equations (5) to (8) described above.

[0096] <Clipping Processing in Lossless Adaptive Color Transformation>

[0097] However, in the case of the above-mentioned lossless YCgCo-R transform, the range of the transformed coefficient data is wider than that in the case of the YCgCo transform. For example, when the coefficient data of the bit depth bitDepth in the RGB domain undergoes the YCgCo-R transform, the range of the coefficient data of the Y component, which is the luminance component, is [-2^bitDepth, 2^bitDepth-1]. The range of the coefficient data of the Cg component and the Co component, which are the color difference components, is [-2^(bitDepth+1), 2^(bitDepth+1)-1]. That is, the dynamic range of the coefficient data of the Y component is bitDepth+1, and the dynamic range of the coefficient data of the Cg component and the Co component is bitDepth+2.

[0098] Therefore, when the method described in Non-Patent Document 4 is applied to the lossless adaptive color conversion disclosed in Non-Patent Document 3, the coefficient data in the YCgCo domain is changed by the clipping process, and there is a possibility that the coefficient data in the RGB domain after the inverse adaptive color conversion will be distorted. Note that this "distortion" refers to the inconsistency with the coefficient data in the RGB domain before the inverse adaptive color conversion, that is, the difference between the coefficient data in the RGB domain before and after the inverse adaptive color conversion. In other words, with such a method, it may be difficult to achieve lossless conversion between the coefficient data in the RGB domain and the coefficient data in the YCgCo domain.

[0099] Figure 1 This diagram illustrates an example of the relationship between input and output values ​​when a residual signal in the RGB domain undergoes a YCgCo transform or a YCgCo-R transform. It is assumed that the bit depth of the coefficient data in the RGB domain, which serves as input for the YCgCo transform and the YCgCo-R transform, is 10 bits (bitDepth = 10). It is assumed that the value range of the coefficient data in the RGB domain is [-2^bitDepth, 2^bitDepth - 1].

[0100] Figure 1 A shows an example of the relationship between the G component (input value) and the Y component (output value). Figure 1 B shows an example of the relationship between the B component (input value) and the Cg component (output value). Figure 1 C shows an example of the relationship between the R component (input value) and the Co component (output value).

[0101] exist Figure 1In the figure, gray circles represent an example of the relationship between input and output values ​​in the case of YCgCo conversion. White circles represent an example of the relationship between input and output values ​​in the case of YCgCo-R conversion. The solid bold line represents an example of the upper bound of clipping. The dotted bold line represents an example of the lower bound of clipping. Here, the upper and lower bounds of the clipping process described in Non-Patent Document 4 are shown.

[0102] like Figure 1 B and Figure 1 As shown in C, it can be seen that after the YCgCo-R transform, the coefficient data of the Cg component and the coefficient data of the Co component may exceed the upper and lower bounds of the clipping. In this case, through the clipping process, values ​​equal to or greater than the upper bound are clipped to the upper bound, and values ​​equal to or less than the lower bound are clipped to the lower bound. In other words, the values ​​of some coefficient data are changed. Therefore, there is a possibility that the coefficient data after the inverse YCgCo-R transform will be distorted (inconsistent with the coefficient data before the YCgCo-R transform) and increase.

[0103] Because inverse adaptive color transform is lossy as described above, there is a possibility that lossless encoding may be difficult to achieve in encoding to which inverse adaptive color transform is applied. In addition, as the distortion of coefficient data in the RGB domain after inverse adaptive color transform increases, there is a possibility that the difference between the decoded image and the image before encoding increases, and the quality of the decoded image may decrease.

[0104] <Setting the range of clipping processing for lossless adaptive color conversion>

[0105] Therefore, coefficient data subjected to adaptive color transform by a lossless method as input to inverse adaptive color transform by a lossless method is clipped at a level based on the bit depth of the coefficient data.

[0106] For example, in the information processing method, coefficient data is clipped at a level based on the bit depth of coefficient data subjected to adaptive color transform by a lossless method, and the coefficient data clipped at the level is subjected to inverse adaptive color transform by a lossless method.

[0107] For example, an information processing device includes: a cropping processing unit that crops coefficient data at a level based on the bit depth of the coefficient data subjected to adaptive color transformation by a lossless method; and an inverse adaptive color transformation unit that performs inverse adaptive color transformation on the coefficient data cropped at the level by the cropping processing unit by a lossless method.

[0108] In this way, clipping can be performed on the input of the lossless inverse adaptive color transform at a level that does not increase the distortion of the coefficient data after the inverse adaptive color transform. Therefore, it is possible to suppress an increase in the load of the inverse adaptive color transform process while suppressing an increase in the distortion of the coefficient data after the inverse adaptive color transform.

[0109] In particular, by setting the upper and lower bounds so that the range of values ​​between the upper and lower bounds of the clipping process includes the range of values ​​that can theoretically be obtained by coefficient data subjected to adaptive color conversion using a lossless method (i.e., making the coefficient data wider than the theoretically possible range of values), it is possible to suppress the occurrence of distortion in the coefficient data after inverse adaptive color conversion. That is, while achieving lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain, it is possible to suppress an increase in the load of the inverse adaptive color conversion process.

[0110] <Method 1>

[0111] In the clipping process, the luminance component and color components (color difference components) of coefficient data subjected to adaptive color transformation using a lossless method can be clipped at the same level. The coefficient data subjected to adaptive color transformation using a lossless method includes the Y component as the luminance component, the Cg component as the color component (color difference component), and the Co component as the color component (color difference component). In other words, all of these components can be clipped at the same level. This makes it easier to perform the clipping process and suppresses any increase in the load compared to a case where clipping is performed at a different level for each component.

[0112] <Method 1-1>

[0113] For example, the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method may be clipped with a value obtained by subtracting 1 from a power of 2 obtained by adding 1 to the bit depth of the coefficient data as an exponent as an upper bound. Furthermore, the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method may be clipped with a value obtained by multiplying -1 by a power of 2 obtained by adding 1 to the bit depth of the coefficient data as an exponent as a lower bound. Furthermore, both upper and lower bound clipping may be performed.

[0114] For example, an upper bound actResMax of the clipping process performed on coefficient data subjected to adaptive color transform by a lossless method is set as in the following equation (16) using a value obtained by adding 1 to the bit depth of the coefficient data. Furthermore, a lower bound actResMin of the clipping process performed on coefficient data subjected to adaptive color transform by a lossless method is set as in the following equation (17) using a value obtained by adding 1 to the bit depth of the coefficient data.

[0115] actResMax=1<<(BitDepth+1)-1

[0116] …(16)

[0117] actResMin=-1<<(BitDepth+1)

[0118] …(17)

[0119] That is, the upper bound actResMax is a value obtained by subtracting 1 from the power of 2 with the value obtained by adding 1 to the bit depth as the exponent. Furthermore, the lower bound actResMin is a value obtained by multiplying -1 by the power of 2 with the value obtained by adding 1 to the bit depth as the exponent. Then, as shown in the following equations (18) to (20), the luminance component and the color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method are clipped using the upper bound actResMax and the lower bound actResMin.

[0120] r Y [x][y]=Clip3(actResMin, actResMax, r Y [x][y])

[0121] …(18)

[0122] r Cb [x][y]=Clip3(actResMin, actResMax, r Cb [x][y])

[0123] …(19)

[0124] r Cr [x][y]=Clip3(actResMin, actResMax, r Cr [x][y])

[0125] …(20)

[0126] The coefficient data clipped in this manner is then subjected to inverse YCgCo-R transform as in the above-mentioned equations (12) to (15) to obtain coefficient data in the RGB domain.

[0127] In this way, the value range between the upper and lower bounds of the clipping process can be made wider than the theoretically possible value range of coefficient data subjected to adaptive color transformation using a lossless method. Therefore, by clipping using such upper and lower bounds, the occurrence of distortion in the coefficient data after inverse adaptive color transformation can be suppressed. In other words, while achieving lossless transformation between coefficient data in the RGB domain and coefficient data in the YCgCo domain, an increase in the load of the inverse adaptive color transformation process can be suppressed.

[0128] <Method 1-2>

[0129] Furthermore, clipping can be performed while taking into account the dynamic range of the buffer. For example, during clipping, coefficient data subjected to adaptive color transformation using a lossless method can be clipped at a level based on the bit depth and the dynamic range of the buffer holding the coefficient data during inverse adaptive color transformation. By clipping the coefficient data in this way while taking into account the dynamic range of the buffer (hardware limitations), the occurrence of buffer overflow can be suppressed. In other words, since the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data to be held, an increase in the dynamic range of the buffer can be suppressed, and an increase in cost can be suppressed.

[0130] For example, the level to be clipped may be a value obtained using the smaller of the following values: a value based on the bit depth of the coefficient data subjected to adaptive color transform using a lossless method rather than the dynamic range of the buffer holding the coefficient data, and a value based on the dynamic range of the buffer rather than the bit depth. As described above, by setting the upper and lower bounds using smaller values ​​(i.e., the clipping process causes the coefficient data to have a narrower value range), it is possible to suppress an increase in the load of the inverse adaptive color transform process while satisfying the hardware limitations of the buffer and suppressing an increase in distortion of the coefficient data after the inverse adaptive color transform.

[0131] For example, the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method may be clipped using a value obtained by subtracting 1 from the smaller value of the power exponent of 2, which is obtained by adding 1 to the bit depth of the coefficient data and subtracting 1 from the dynamic range of the buffer holding the coefficient data, as an upper bound. Furthermore, the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method may be clipped using a value obtained by multiplying -1 by the smaller value of the power exponent of 2, which is obtained by adding 1 to the bit depth of the coefficient data and subtracting 1 from the dynamic range of the buffer holding the coefficient data, as a lower bound. Furthermore, both upper and lower bound clipping can be performed.

[0132] For example, an upper bound actResMax of the clipping process performed on coefficient data subjected to adaptive color transform by a lossless method is set as in the following equation (21) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data. Furthermore, a lower bound actResMin of the clipping process performed on coefficient data subjected to adaptive color transform by a lossless method is set as in the following equation (22) using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data.

[0133] actResMax=1<<(min(log2MaxDR-1, BitDepth+1))-1

[0134] …(twenty one)

[0135] actResMin=-1<<(min(log2MaxDR-1, BitDepth+1))

[0136] …(twenty two)

[0137] That is, the upper bound actResMax is a value obtained by subtracting 1 from the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to the power of 2. Furthermore, the lower bound actResMin is a value obtained by multiplying -1 by the smaller of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to the power of 2. Then, the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method are clipped using the upper bound actResMax and the lower bound actResMin shown in equations (18) to (20) above.

[0138] The coefficient data clipped in this manner is then subjected to inverse YCgCo-R transform as described above in equations (12) to (15) to obtain coefficient data in the RGB domain.

[0139] In this way, it is possible to suppress an increase in the load of the inverse adaptive color transform process while satisfying the hardware limitation of the buffer and suppressing an increase in distortion of coefficient data after the inverse adaptive color transform.

[0140] <Method 2>

[0141] As described above in <Clipping Processing in Adaptive Color Transform of Lossless Method>, the value range of coefficient data of the Cg component and the Co component as color components (color difference components) is wider than the value range of coefficient data of the Y component as the brightness component.

[0142] Therefore, the luminance component and color difference component of the coefficient data can be clipped at their respective levels. For example, the luminance component of the coefficient data can be clipped at a first level, and the color component (color difference component) of the coefficient data can be clipped at a second level. In this way, since the coefficient data of each component can be clipped within the value range corresponding to that component, it is possible to further suppress an increase in the load of the inverse adaptive color transform processing while suppressing an increase in distortion of the coefficient data after inverse adaptive color transform.

[0143] For example, when clipping is performed based on a component having a wide value range, the clipping width (between the upper and lower bounds) becomes unnecessarily wide in a component having a narrower range, and the load reduction of the inverse adaptive color conversion process is suppressed. As described above, by clipping each component with a width corresponding to that component, the increase in the load of the inverse adaptive color conversion process can be further suppressed.

[0144] <Method 2-1>

[0145] The difference between the upper and lower bounds of the second level can be wider than the difference between the upper and lower bounds of the first level. As described above, the range of the coefficient data for the Cg and Co components, which are color difference components, is wider than the range of the coefficient data for the Y component, which is the luminance component. Therefore, by making the difference between the upper and lower bounds of the clipping of the coefficient data for the Cg and Co components, which are color difference components, wider than the difference between the upper and lower bounds of the clipping of the coefficient data for the Y component, which is the luminance component, it is possible to further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data after inverse adaptive color conversion.

[0146] For example, the luma component of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by subtracting 1 from a power of 2 exponented by the bit depth of the coefficient data as an upper bound. Furthermore, the luma component of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by multiplying -1 by a power of 2 exponented by the bit depth of the coefficient data as a lower bound. Furthermore, both upper and lower bound clipping of the luma component can be performed. Furthermore, the color component (color difference component) of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by subtracting 1 from a power of 2 exponented by a value obtained by adding 1 to the bit depth of the coefficient data as an upper bound. Furthermore, the color component (color difference component) of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by multiplying -1 by a power of 2 exponented by a value obtained by adding 1 to the bit depth of the coefficient data as a lower bound. In addition, the upper limit clipping and the lower limit clipping can be performed on the color component (color difference component). In addition, as described above, the upper limit clipping and the lower limit clipping can be performed on each of the brightness component and the color component (color difference component).

[0147] For example, the upper bound actResMaxY of the clipping process performed on the luminance component of the coefficient data subjected to adaptive color transformation by the lossless method is set using the bit depth of the coefficient data as in the following equation (23). In addition, the lower bound actResMinY of the clipping process performed on the luminance component of the coefficient data subjected to adaptive color transformation by the lossless method is set using the bit depth of the coefficient data as in the following equation (24). In addition, the upper bound actResMaxC of the clipping process performed on the color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method is set using the bit depth of the coefficient data as in the following equation (25). In addition, the lower bound actResMinC of the clipping process performed on the color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method is set using the bit depth of the coefficient data as in the following equation (26).

[0148] actResMaxY=1<<BitDepth-1

[0149] …(twenty three)

[0150] actResMinY=-1<<BitDepth

[0151] …(twenty four)

[0152] actResMaXC=1<<(BitDepth+1)-1

[0153] …(25)

[0154] actResMinC=-1<<(BitDepth+1)

[0155] …(26)

[0156] That is, the upper bound actResMaxY of the luma component is a value obtained by subtracting 1 from the power of 2 with the bit depth as the exponent. In addition, the lower bound actResMinY of the luma component is a value obtained by multiplying -1 by the power of 2 with the bit depth as the exponent. Then, as expressed in the following equation (27), the luma component of the coefficient data subjected to adaptive color transformation by the lossless method is clipped using the upper bound actResMaxY and the lower bound actResMinY.

[0157] r Y [x][y]=Clip3(actResMinY,actResMaxY,r Y [x][y])

[0158] …(27)

[0159] In addition, the upper bound actResMaxC of the color component (color difference component) is set to a value obtained by subtracting 1 from the power of 2 with the value obtained by adding 1 to the bit depth as the exponent. In addition, the lower bound actResMinC of the color component (color difference component) is set to a value obtained by multiplying -1 by the power of 2 with the value obtained by adding 1 to the bit depth as the exponent. Then, as shown in the following equations (28) and (29), the color component (color difference component) of the coefficient data subjected to adaptive color conversion by the lossless method is clipped using the upper bound actResMaxC and the lower bound actResMinC.

[0160] r Cb [x][y]=Clip3(actResMinC, actResMaxC, r Cb [x][y])

[0161] …(28)

[0162] r Cr [x][y]=Clip3(actResMinC, actResMaxC, r Cr [x][y])

[0163] …(29)

[0164] The coefficient data clipped in this manner is then subjected to the inverse YCgCo-R transform in equations (12) to (15) as described above to yield coefficient data in the RGB domain.

[0165] In this way, since the coefficient data of each component can be clipped within the value range corresponding to each component, it is possible to further suppress the increase in the load of the adaptive color inverse conversion process while suppressing the increase in distortion of the coefficient data after the adaptive color inverse conversion.

[0166] <Method 2-2>

[0167] Furthermore, clipping can be performed while taking into account the dynamic range of the buffer. For example, during clipping, coefficient data subjected to adaptive color transformation using a lossless method can be clipped at a level based on the bit depth and the dynamic range of the buffer holding the coefficient data during inverse adaptive color transformation. That is, the first level and the second level can be values ​​based on the bit depth of the coefficient data subjected to adaptive color transformation using a lossless method and the dynamic range of the buffer storing the coefficient data, respectively. By clipping the coefficient data in this way while taking into account the dynamic range of the buffer (hardware limitations), the occurrence of buffer overflow can be suppressed. That is, since the dynamic range of the buffer can be set without taking into account the bit depth of the retained coefficient data, an increase in the dynamic range of the buffer can be suppressed, and an increase in cost can be suppressed.

[0168] For example, the level of clipping the luma component may be a value derived using the smaller of the following values: a value based on the bit depth of the coefficient data subjected to adaptive color transformation using a lossless method, rather than a value based on the dynamic range of the buffer holding the coefficient data, and a value based on the dynamic range of the buffer, rather than a value based on the bit depth. Furthermore, the level of clipping the color component (color difference component) may be a value derived using the smaller of the following values: a value based on the bit depth of the coefficient data subjected to adaptive color transformation using a lossless method, plus 1, rather than a value based on the dynamic range of the buffer holding the coefficient data, and a value based on the dynamic range of the buffer, rather than a value based on the bit depth plus 1. As described above, by setting the upper and lower bounds using smaller values ​​(i.e., the clipping process results in a narrower value range for the coefficient data), it is possible to further suppress an increase in the load of the inverse adaptive color transformation process while satisfying the hardware limitations of the buffer and suppressing an increase in distortion of each component of the coefficient data after inverse adaptive color transformation.

[0169] For example, the luma component of the coefficient data may be clipped using a value obtained by subtracting 1 from the smaller of the value obtained by subtracting 1 from the bit depth of the coefficient data subjected to adaptive color transformation using a lossless method and the dynamic range of the buffer holding the coefficient data, raising the power of 2 as the exponent. Furthermore, the luma component of the coefficient data may be clipped using a value obtained by multiplying -1 by the smaller of the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data and raising the power of 2 as the exponent as the lower bound. Furthermore, both the upper and lower bounds of the luma component may be clipped in this manner.

[0170] Furthermore, the color components (color difference components) of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by subtracting 1 from the smaller of the value obtained by adding 1 to the bit depth of the coefficient data and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to a power of 2, as an upper bound. Furthermore, the color components (color difference components) of coefficient data subjected to lossless adaptive color transformation can be clipped using a value obtained by multiplying -1 by the smaller of the value obtained by adding 1 to the bit depth of the coefficient data and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to a power of 2, as a lower bound. Both upper and lower bound clipping can be performed on the color components (color difference components). Furthermore, as described above, both upper and lower bound clipping can be performed on each of the luminance component and the color components (color difference components).

[0171] For example, an upper bound actResMaxY of the clipping process performed on the luminance component of the coefficient data subjected to adaptive color transform by the lossless method is set using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data as in the following equation (30). Furthermore, a lower bound actResMinY of the clipping process performed on the luminance component of the coefficient data subjected to adaptive color transform by the lossless method is set using the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data as in the following equation (31). Furthermore, an upper bound actResMaxC of the clipping process performed on the color component (color difference component) of the coefficient data subjected to adaptive color transform by the lossless method is set using a value obtained by adding 1 to the bit depth of the coefficient data and a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data as in the following equation (32). In addition, the lower bound actResMinC of the clipping processing performed on the color components (color difference components) of the coefficient data that undergoes adaptive color transformation by the lossless method is set as in the following equation (33) using the value obtained by adding 1 to the bit depth of the coefficient data and the value obtained by subtracting 1 from the dynamic range of the buffer that holds the coefficient data.

[0172] actResMaxY=1<<(min(log2MaxDR-1,BitDepth))-1

[0173] …(30)

[0174] actResMinY=-1<<(min(log2MaxDR-1,BitDepth))

[0175] …(31)

[0176] actResMaxC=1<<(min(log2MaxDR-1, BitDepth+1))-1

[0177] …(32)

[0178] actResMinC=-1<<(min(log2MaxDR-1,BitDepth+1))

[0179] …(33)

[0180] That is, the upper bound actResMaxY of the luminance component is a value obtained by subtracting 1 from the power of 2 with the smaller value of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data as the exponent. In addition, the lower bound actResMinY of the luminance component is a value obtained by multiplying -1 by the power of 2 with the smaller value of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data as the exponent. Then, as shown in the above equation (27), the luminance component of the coefficient data subjected to adaptive color transformation by the lossless method is clipped using the upper bound actResMaxY and the lower bound actResMinY.

[0181] In addition, the upper bound actResMaxC of the color component (color difference component) is a value obtained by subtracting 1 from the smaller value of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to the power of 2. Furthermore, the lower bound actResMinC of the color component (color difference component) is set to a value obtained by multiplying -1 by the smaller value of the value obtained by adding 1 to the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data, raised to the power of 2. Then, as shown in the above equations (28) and (29), the color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method is clipped using the upper bound actResMaxC and the lower bound actResMinC.

[0182] The coefficient data clipped in this manner is then subjected to inverse YCgCo-R transform as in equations (12) to (15) above to yield coefficient data in the RGB domain.

[0183] In this way, it is possible to further suppress an increase in the load of the inverse adaptive color transform process while satisfying the hardware limitation of the buffer and suppressing an increase in distortion of each component of coefficient data after inverse adaptive color transform.

[0184] <Method 3>

[0185] The coefficient data can be clipped at a level based on the dynamic range of the buffer holding the coefficient data during inverse adaptive color transform. By clipping the coefficient data in this way while taking into account the dynamic range of the buffer (hardware limitations), the occurrence of buffer overflow can be suppressed. In other words, since the dynamic range of the buffer can be set without taking into account the bit depth of the coefficient data to be held, an increase in the dynamic range of the buffer can be suppressed, and an increase in cost can be suppressed.

[0186] For example, the luminance component and color component (color difference component) of the coefficient data may be clipped using as an upper bound a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data at the time of inverse adaptive color transform, raised to the power of 2, as a power exponent. Furthermore, the luminance component and color component (color difference component) of the coefficient data may be clipped using as a lower bound a value obtained by multiplying -1 by a value obtained by subtracting 1 from the dynamic range of the buffer, raised to the power of 2, as a power exponent.

[0187] For example, an upper bound actResMax of the clipping process performed on coefficient data subjected to adaptive color conversion by a lossless method is set using a value obtained by subtracting 1 from the dynamic range of the buffer as in the following equation (34). Furthermore, a lower bound actResMin of the clipping process performed on coefficient data subjected to adaptive color conversion by a lossless method is set using a value obtained by subtracting 1 from the dynamic range of the buffer as in the following equation (35).

[0188] actResMax=1<<(log2MaxDR-1)-1

[0189] …(34)

[0190] actResMin=-1<<(log2MaxDR-1)

[0191] …(35)

[0192] That is, the upper bound actResMax is a value obtained by subtracting 1 from the power of 2, with the value obtained by subtracting 1 from the dynamic range of the buffer as the exponent. In addition, the lower bound actResMin is a value obtained by multiplying -1 by the power of 2, with the value obtained by subtracting 1 from the dynamic range of the buffer as the exponent. Then, as shown in the above equations (18) to (20), the luminance component and color component (color difference component) of the coefficient data subjected to adaptive color transformation by the lossless method are clipped using the upper bound actResMax and the lower bound actResMin.

[0193] The coefficient data clipped in this manner is then subjected to inverse YCgCo-R transform as in equations (12) to (15) above to yield coefficient data in the RGB domain.

[0194] In this way, the hardware limitations of the buffer can be met and the occurrence of buffer overflow can be suppressed. Furthermore, an increase in cost can be suppressed. Furthermore, since the value range of the coefficient data is limited by the clipping process, an increase in the load of the inverse adaptive color transform process can be suppressed. Furthermore, in the case where the value range of the coefficient data is narrower than the dynamic range of the buffer, this method can suppress an increase in distortion of the coefficient data after the inverse adaptive color transform.

[0195] <Application Examples>

[0196] In addition, the above-mentioned "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3" can be selectively applied. For example, the optimal method can be applied according to predetermined application conditions based on certain information such as input coefficient data, hardware specifications, and load conditions. For example, in the case where the dynamic range of the buffer is wider than the value range of the coefficient data, "Method 1-1" or "Method 2-1" can be applied. When the value range of the luminance component and the value range of the color component (color difference component) of the coefficient data are equal to each other, "Method 1", "Method 1-1", or "Method 1-2" can be applied.

[0197] In addition, in each method, either or both of upper and lower bound clipping can be selectively applied. For example, when the lower bound of clipping is equal to or less than the lower bound of the value range of the coefficient data, lower bound clipping can be omitted (skipped). In addition, when the upper bound of clipping is equal to or greater than the upper bound of the value range of the coefficient data, upper bound clipping can be omitted (skipped).

[0198] Of course, these are examples, and the application conditions or methods applied to each condition are not limited to these examples.

[0199] <2. First embodiment>

[0200] <Inverse Adaptive Color Conversion Device>

[0201] The present technology described above can be applied to any device. Figure 2 : is a block diagram showing an example of the configuration of an inverse adaptive color conversion device as one aspect of an image processing device to which the present technology is applied. Figure 2 The inverse adaptive color conversion device 100 shown is a device that performs inverse adaptive color conversion (inverse YCgCo-R conversion) by a lossless method on coefficient data in the YCgCo domain, where coefficient data in the RGB domain associated with an image has been subjected to adaptive color conversion (YCgCo-R conversion) by a lossless method.

[0202] Note that in Figure 2 In the figure, the main processing units, data flow, etc. are shown, and Figure 2That is, in the inverse adaptive color conversion device 100, there may be Figure 2 The processing units shown as blocks in FIG, or there may be processing units not shown in FIG. Figure 2 The process or data flow is shown as arrows etc.

[0203] like Figure 2 As shown, the inverse adaptive color conversion apparatus 100 includes a selection unit 101 , a cropping processing unit 102 , and an inverse YCgCo-R conversion unit 103 .

[0204] The selection unit 101 obtains the coefficient data res_x' input to the inverse adaptive color conversion device 100. Furthermore, the selection unit 101 obtains the cu_act_enabled_flag input to the inverse adaptive color conversion device 100. The cu_act_enabled_flag is flag information indicating whether adaptive color conversion (inverse adaptive color conversion) can be applied. When the cu_act_enabled_flag is true (e.g., "1"), it indicates that the adaptive color conversion (inverse adaptive color conversion) can be applied. When the cu_act_enabled_flag is false (e.g., "0"), it indicates that the application of the adaptive color conversion (inverse adaptive color conversion) is prohibited (i.e., not applicable).

[0205] The selection unit 101 selects whether to perform inverse adaptive color transform on the coefficient data res_x' based on cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (e.g., "1"), the selection unit 101 determines that the coefficient data res_x' is coefficient data in the YCgCo domain that has been subjected to YCgCo-R transform on the coefficient data in the RGB domain. The selection unit 101 then supplies the coefficient data res_x' to the cropping processing unit 102 to perform inverse YCgCo-R transform on the coefficient data res_x'.

[0206] For example, when cu_act_enabled_flag is false (e.g., "0"), the selection unit 101 determines that the coefficient data res_x' is coefficient data in the RGB domain. Then, the selection unit 101 outputs the coefficient data res_x' as the coefficient data res_x after the inverse adaptive color conversion to the outside of the inverse adaptive color conversion device 100. In other words, the coefficient data in the RGB domain is output to the outside of the inverse adaptive color conversion device 100.

[0207] The cropping unit 102 receives the coefficient data res_x' supplied from the selection unit 101. Furthermore, the cropping unit 102 receives variables input to the inverse adaptive color conversion apparatus 100, such as log2MaxDR and BitDepth. Log2MaxDR represents the dynamic range of the buffer holding the coefficient data res_x' during the inverse adaptive color conversion process. BitDepth represents the bit depth of the coefficient data res_x'.

[0208] The cropping processing unit 102 performs a cropping process on the coefficient data res_x′ by using upper and lower bounds derived based on variables such as log2MaxDR and BitDepth, and supplies the cropped coefficient data res_x′ to the inverse YCgCo-R conversion unit 103 .

[0209] The inverse YCgCo-R conversion unit 103 obtains the coefficient data res_x' subjected to the cropping process, which is provided from the cropping processing unit 102. The inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the obtained coefficient data res_x' to generate coefficient data res_x subjected to the inverse YCgCo-R conversion. The inverse YCgCo-R conversion unit 103 outputs the generated coefficient data res_x to the outside of the inverse adaptive color conversion device 100. In other words, the coefficient data in the RGB domain is output to the outside of the inverse adaptive color conversion device 100.

[0210] <Application of this technology to inverse adaptive color conversion device>

[0211] In such an inverse adaptive color conversion device 100 , the present technology described above in <1. Clipping Processing of Lossless Inverse Adaptive Color Conversion> can be applied.

[0212] For example, the cropping processing unit 102 crops the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R conversion unit 103 performs inverse adaptive color conversion on the coefficient data res_x' cropped at the level by the cropping processing unit 102 using a lossless method.

[0213] In this way, the cropping processing unit 102 can crop the coefficient data res_x' to a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color conversion. Therefore, the inverse adaptive color conversion device 100 can suppress the increase in the load of the inverse adaptive color conversion process while suppressing the increase in the distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0214] Note that the inverse adaptive color conversion device 100 can apply the various methods of the present technology described in the above <1. Clipping processing of lossless inverse adaptive color conversion> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2" and "Method 3").

[0215] For example, as described in <Method 1> above, the cropping processing unit 102 may crop the luminance component and the color component (color difference component) of the coefficient data res_x′ at the same level.

[0216] Then, the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped in this manner to obtain coefficient data res_x. In this way, the clipping processing unit 102 can easily perform clipping processing and suppress an increase in load compared to the case where clipping is performed at different levels for each component.

[0217] For example, as described above in <Method 1-1>, the clipping processing unit 102 can clip the luminance component and color component (color difference component) of the coefficient data res_x' using a value obtained by subtracting 1 from the power of 2 obtained by adding 1 to the bit depth of the coefficient data res_x' as the power exponent as the upper bound. In addition, the clipping processing unit 102 can clip the luminance component and color component (color difference component) of the coefficient data res_x' using a value obtained by multiplying -1 by the power of 2 obtained by adding 1 to the bit depth of the coefficient data res_x' as the power exponent as the lower bound. In addition, the clipping processing unit 102 can perform both such upper bound clipping and lower bound clipping.

[0218] The inverse YCgCo-R conversion unit 103 then performs an inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, yielding coefficient data res_x. In this manner, the clipping unit 102 can make the range between the upper and lower bounds of the clipping process wider than the theoretically possible range of the coefficient data res_x'. Consequently, the inverse adaptive color conversion apparatus 100 can suppress distortion of the coefficient data res_x after the inverse adaptive color conversion. In other words, the inverse adaptive color conversion apparatus 100 can achieve lossless conversion between coefficient data in the RGB domain and coefficient data in the YCgCo domain while suppressing an increase in the load of the inverse adaptive color conversion process.

[0219] Furthermore, for example, as described above in <Method 1-2>, the cropping processing unit 102 may crop the coefficient data res_x' at a level based on the bit depth and the dynamic range of the buffer holding the coefficient data res_x' during inverse adaptive color transform.

[0220] The inverse YCgCo-R conversion unit 103 then performs an inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, thereby obtaining coefficient data res_x. By clipping the coefficient data res_x' in this manner while taking into account the dynamic range of the buffer (hardware limitations), the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data res_x' to be stored, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color conversion device 100 can be suppressed.

[0221] For example, the cropping processing unit 102 may perform cropping processing at a level derived using the smaller of a value based on the bit depth of the coefficient data res_x' rather than the dynamic range of the buffer holding the coefficient data res_x' and a value based on the dynamic range of the buffer rather than the bit depth.

[0222] The inverse YCgCo-R conversion unit 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped to obtain coefficient data res_x. As described above, by setting the upper and lower bounds to smaller values ​​(i.e., clipping the coefficient data res_x' to have a narrower value range), the inverse adaptive color conversion apparatus 100 can suppress an increase in the load of the inverse adaptive color conversion process while satisfying the hardware limitations of the buffer and suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0223] For example, the clipping processing unit 102 can clip the luminance component and color component (color difference component) of the coefficient data res_x' using a value obtained by subtracting 1 from the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as a power of 2 as the power exponent as an upper bound. In addition, the clipping processing unit 102 can clip the luminance component and color component (color difference component) of the coefficient data res_x' using a value obtained by multiplying -1 by the smaller of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as a power of 2 as the power exponent as a lower bound. In addition, the clipping processing unit 102 can perform both such upper bound clipping and lower bound clipping.

[0224] The inverse YCgCo-R conversion unit 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped to obtain coefficient data res_x. In this manner, the inverse adaptive color conversion apparatus 100 can suppress an increase in the load of the inverse adaptive color conversion process while satisfying hardware limitations of the buffer and suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0225] For example, as described above in <Method 2>, the cropping processing unit 102 can crop the luminance component and color difference component of the coefficient data res_x' at their respective levels (first level and second level). In this way, the cropping processing unit 102 can crop the coefficient data of each component within the value range corresponding to that component. The inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the coefficient data res_x' thus cropped, and obtains coefficient data res_x. Therefore, the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0226] For example, as described above in <Method 2-1>, the difference between the upper and lower bounds of the second level can be wider than the difference between the upper and lower bounds of the first level. The inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x' clipped in this manner, and obtains coefficient data res_x. By making the difference between the clipped upper and lower bounds of the coefficient data res_x' for the Cg and Co components, which are color difference components, wider than the difference between the clipped upper and lower bounds of the coefficient data res_x' for the Y component, which is the luminance component, the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0227] For example, the clipping processing unit 102 may clip the luma component of the coefficient data res_x' using a value obtained by subtracting 1 from the power of 2 with the bit depth of the coefficient data res_x' as an exponent as an upper bound. Furthermore, the luma component of the coefficient data res_x' may clip the luma component of the coefficient data res_x' using a value obtained by multiplying -1 by the power of 2 with the bit depth of the coefficient data res_x' as an exponent as a lower bound. Furthermore, the clipping processing unit 102 may perform both such upper bound clipping and lower bound clipping on the luma component.

[0228] In addition, the clipping processing unit 102 can clip the color component (color difference component) of the coefficient data res_x' using a value obtained by subtracting 1 from the power of 2 obtained by adding 1 to the bit depth of the coefficient data res_x' as an upper bound. In addition, the color component (color difference component) of the coefficient data res_x' can be clipped using a value obtained by multiplying -1 by the power of 2 obtained by adding 1 to the bit depth of the coefficient data res_x' as an exponent as a lower bound. In addition, the clipping processing unit 102 can perform both such upper bound clipping and lower bound clipping on the color component (color difference component). In addition, as described above, the clipping processing unit 102 can perform both upper bound clipping and lower bound clipping on each of the luminance component and the color component (color difference component).

[0229] The inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the luminance component and color components (color difference components) of the coefficient data res_x' thus clipped, thereby obtaining coefficient data res_x. In this manner, the clipping processing unit 102 can clip the coefficient data res_x' for each component within the value range corresponding to that component. Therefore, the inverse adaptive color conversion apparatus 100 can further suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion of the coefficient data res_x after the inverse adaptive color conversion.

[0230] Furthermore, for example, as described above in <Method 2-2>, the cropping processing unit 102 may crop the coefficient data res_x' at levels (first and second levels) based on the bit depth and the dynamic range of the buffer holding the coefficient data res_x' during inverse adaptive color transform.

[0231] The inverse YCgCo-R transform unit 103 performs an inverse YCgCo-R transform on the luminance component and color component (color difference component) of the coefficient data res_x' thus clipped, thereby obtaining the coefficient data res_x. In this manner, the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set regardless of the bit depth of the coefficient data res_x' to be stored, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color conversion device 100 can be suppressed.

[0232] For example, the cropping processing unit 102 may perform a cropping process on the luminance component of the coefficient data res_x' at a level derived using a smaller value of the following values: a value based on the bit depth of the coefficient data res_x' instead of a value based on the dynamic range of the buffer that maintains the coefficient data res_x', and a value based on the dynamic range of the buffer instead of the bit depth. In addition, the cropping processing unit 102 may perform a cropping process on the color component (color difference component) of the coefficient data res_x' using a level derived using a smaller value of the following values: a value based on the bit depth of the coefficient data res_x' plus 1 instead of a value based on the dynamic range of the buffer that maintains the coefficient data res_x', and a value based on the dynamic range of the buffer instead of the value based on the bit depth plus 1.

[0233] Then, the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the coefficient data res_x' clipped in this manner to obtain coefficient data res_x. As described above, by setting the upper and lower bounds using smaller values ​​(i.e., clipping the coefficient data so that the value range is narrower), the inverse adaptive color conversion device 100 can further suppress an increase in the load of the inverse adaptive color conversion process while satisfying the hardware limitations of the buffer and suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color conversion.

[0234] For example, the clipping processing unit 102 may clip the luma component of the coefficient data res_x' using a value obtained by subtracting 1 from the power of 2 with the smaller value of the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as the power exponent as an upper bound. In addition, the clipping processing unit 102 may clip the luma component of the coefficient data res_x' using a value obtained by multiplying -1 by the power of 2 with the smaller value of the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as the power exponent as a lower bound. In addition, the clipping processing unit 102 may perform both such upper bound clipping and lower bound clipping on the luma component.

[0235] In addition, the clipping processing unit 102 can clip the color component (color difference component) of the coefficient data res_x' using a value obtained by subtracting 1 from the smaller value of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as a power of 2 as the power exponent as the lower limit. In addition, the clipping processing unit 102 can clip the color component (color difference component) of the coefficient data res_x' using a value obtained by multiplying -1 by the smaller value of the value obtained by adding 1 to the bit depth of the coefficient data res_x' and the value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' as a power of 2 as the power exponent as the lower limit. In addition, the clipping processing unit 102 can perform both such upper limit clipping and lower limit clipping on the color component (color difference component). Furthermore, as described above, the clipping processing unit 102 can perform upper bound clipping and lower bound clipping on each of the luminance component and the color component (color difference component).

[0236] The inverse YCgCo-R conversion unit 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' thus clipped to obtain coefficient data res_x. In this manner, the inverse adaptive color conversion apparatus 100 can further suppress an increase in the load of the inverse adaptive color conversion process while satisfying hardware limitations of the buffer and suppressing an increase in distortion of each component of the coefficient data res_x after the inverse adaptive color conversion.

[0237] For example, as described above in <Method 3>, the cropping processing unit 102 may crop the coefficient data res_x' at a level based on the dynamic range of the buffer holding the coefficient data res_x' during inverse adaptive color transform.

[0238] The inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the coefficient data res_x' clipped in this manner to obtain coefficient data res_x. In this manner, the inverse adaptive color conversion device 100 can suppress the occurrence of buffer overflow. In other words, since the dynamic range of the buffer can be set without considering the bit depth of the coefficient data to be stored, an increase in the dynamic range of the buffer can be suppressed, and an increase in the cost of the inverse adaptive color conversion device 100 can be suppressed.

[0239] For example, the clipping processing unit 102 can clip the luminance component and the color component (color difference component) of the coefficient data res_x' using a value obtained by subtracting 1 from the power of 2 with a value obtained by subtracting 1 from the dynamic range of the buffer holding the coefficient data res_x' at the time of inverse adaptive color transform as an upper bound. In addition, the clipping processing unit 102 can clip the luminance component and the color component (color difference component) of the coefficient data res_x' using a value obtained by multiplying -1 by a value obtained by subtracting 1 from the dynamic range of the buffer as a power of 2 with a power exponent as a lower bound.

[0240] The inverse YCgCo-R conversion unit 103 then performs an inverse YCgCo-R conversion on the coefficient data res_x' thus clipped, yielding coefficient data res_x. In this manner, the inverse adaptive color conversion apparatus 100 can meet the hardware limitations of the buffer and suppress buffer overflow. Furthermore, an increase in the cost of the inverse adaptive color conversion apparatus 100 can be suppressed. Furthermore, since the value range of the coefficient data is limited by the clipping process, the inverse adaptive color conversion apparatus 100 can suppress an increase in the load of the inverse adaptive color conversion process. Furthermore, in cases where the value range of the coefficient data is narrower than the dynamic range of the buffer, the inverse adaptive color conversion apparatus 100 can suppress an increase in distortion of the coefficient data after inverse adaptive color conversion.

[0241] Note that the inverse adaptive color conversion device 100 can apply various application examples described above in <Application Examples>.

[0242] <Flow of Inverse Adaptive Color Conversion Processing>

[0243] Next, we will refer to Figure 3 The flowchart of FIG. 1 describes an example of the flow of the inverse adaptive color conversion process performed by the inverse adaptive color conversion apparatus 100 .

[0244] When the inverse adaptive color conversion process is started, in step S101, the selection unit 101 of the inverse adaptive color conversion device 100 determines whether cu_act_enabled_flag is true. In the case where it is determined that cu_act_enabled_flag is true, the process proceeds to step S102.

[0245] In step S102 , the cropping processing unit 102 crops the coefficient data res_x′ using a predetermined upper limit and a predetermined lower limit.

[0246] In step S103 , the inverse YCgCo-R transform unit 103 performs inverse YCgCo-R transform on the coefficient data res_x′ clipped in step S102 to obtain inverse adaptive color transformed coefficient data res_x.

[0247] In step S104, the inverse YCgCo-R conversion unit 103 outputs the obtained coefficient data res_x to the outside of the inverse adaptive color conversion device 100. When the process of step S104 ends, the inverse adaptive color conversion process ends.

[0248] Furthermore, in step S101, in a case where it is determined that cu_act_enabled_flag is false, the process proceeds to step S105.

[0249] In step S105, the selection unit 101 outputs the coefficient data res_x' as coefficient data res_x after inverse adaptive color conversion to the outside of the inverse adaptive color conversion device 100. When the process of step S105 ends, the inverse adaptive color conversion process ends.

[0250] <Application of this technology in inverse adaptive color transformation processing>

[0251] In such an inverse adaptive color transform process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied.

[0252] For example, in step S102, the cropping processing unit 102 crops the coefficient data res_x' at a level based on the bit depth of the coefficient data. Then, in step S103, the inverse YCgCo-R conversion unit 103 performs inverse adaptive color conversion on the coefficient data res_x' cropped at the level by the cropping processing unit 102 using a lossless method.

[0253] In this way, the cropping unit 102 can "crop" the coefficient data res_x to a level that does not increase distortion in the coefficient data res_x after inverse adaptive color conversion. Therefore, the inverse adaptive color conversion apparatus 100 can suppress an increase in distortion in the coefficient data res_x after inverse adaptive color conversion while also suppressing an increase in the load of the inverse adaptive color conversion process.

[0254] Note that in such inverse adaptive color conversion processing, various methods of the present technology described above in <1. Cropping Processing of Lossless Inverse Adaptive Color Conversion> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," and "Method 2") can be applied. By applying any of these methods, effects similar to those described in <Application of the Present Technology to an Inverse Adaptive Color Conversion Device> can be obtained.

[0255] <3. Second embodiment>

[0256] <Inverse Quantization and Inverse Transformation Device>

[0257] The above-described inverse adaptive color transform apparatus 100 (inverse adaptive color transform processing) can be applied to an inverse quantization and inverse transform apparatus that performs an inverse quantization and inverse transform processing. Figure 4 : is a block diagram showing an example of the configuration of an inverse quantization and inverse transformation device as one aspect of an image processing device to which the present technology is applied. Figure 4 The inverse quantization and inverse transform device 200 shown is a device that performs adaptive color transform (YCgCo-R transform) on coefficient data in the RGB domain associated with an image using a lossless method, performs orthogonal transform, inversely quantizes the quantized coefficients quantized using a method corresponding to quantization, performs inverse orthogonal transform using a method corresponding to orthogonal transform, and performs inverse adaptive color transform (inverse YCgCo-R transform) using a lossless method. Such processing (processing performed by the inverse quantization and inverse transform device 200) is also referred to as inverse quantization and inverse transform processing.

[0258] Note that in Figure 4 In the figure, the main processing units, data flow, etc. are shown, and Figure 4 That is, in the inverse quantization and inverse transformation device 200, there may be Figure 4 The processing units shown as blocks in FIG, or there may be processing units not shown in FIG. Figure 4 The process or data flow is shown as arrows etc.

[0259] like Figure 4 As shown, the inverse quantization and inverse transformation apparatus 200 includes an inverse quantization unit 201 , an inverse orthogonal transformation unit 202 and an inverse adaptive color transformation unit 203 .

[0260] The inverse quantization unit 201 obtains the quantized coefficient qcoef_x. The quantized coefficient qcoef_x is obtained by quantizing the orthogonal transform coefficient coef_x using a predetermined method. Furthermore, the inverse quantization unit 201 obtains parameters required for inverse quantization, such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag. transform_skip_flag is flag information indicating whether to skip (omit) the inverse orthogonal transform process. For example, when transform_skip_flag is true, the inverse orthogonal transform process is skipped. Furthermore, when transform_skip_flag is false, the inverse orthogonal transform process is performed.

[0261] Using these parameters, the inverse quantization unit 201 inversely quantizes the quantized coefficient qcoef_x by a predetermined method corresponding to the above quantization to obtain an orthogonal transform coefficient coef_x. The inverse quantization unit 201 supplies the obtained orthogonal transform coefficient coef_x to the inverse orthogonal transform unit 202.

[0262] The inverse orthogonal transform unit 202 obtains the orthogonal transform coefficient coef_x provided by the inverse quantization unit 201. The orthogonal transform coefficient coef_x is obtained by performing an orthogonal transform on the coefficient data res_x' using a predetermined method. Furthermore, the inverse orthogonal transform unit 202 obtains parameters required for inverse quantization, such as transform information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx. mts_idx is an identifier for the Multiple Transform Select (MTS). lfnst_idx is mode information related to the low-frequency secondary transform. Using these parameters, the inverse orthogonal transform unit 202 performs an inverse orthogonal transform on the orthogonal transform coefficient coef_x using a predetermined method corresponding to the above-mentioned orthogonal transform, and obtains coefficient data res_x' that has been adaptively color transformed using a lossless method. The inverse orthogonal transform unit 202 provides the obtained coefficient data res_x' to the inverse adaptive color transform unit 203.

[0263] The inverse adaptive color transform unit 203 obtains the coefficient data res_x' supplied from the inverse orthogonal transform unit 202. The coefficient data res_x' is as described in the first embodiment. Furthermore, the inverse adaptive color transform unit 203 obtains the cu_act_enabled_flag. Based on the cu_act_enabled_flag, the inverse adaptive color transform unit 203 appropriately performs inverse adaptive color transform (inverse YCgCo-R transform) on the coefficient data res_x' using a lossless method to obtain coefficient data res_x after the inverse adaptive color transform process. The coefficient data res_x is as described in the first embodiment and is coefficient data in the RGB domain.

[0264] The inverse adaptive color conversion unit 203 outputs the obtained coefficient data res_x to the outside of the inverse quantization and inverse conversion device 200.

[0265] <Application of this technology in inverse quantization and inverse transformation devices>

[0266] In such an inverse quantization and inverse transformation device 200, the present technology described above in <1. Clipping Processing of Lossless Inverse Adaptive Color Transform> can be applied. That is, the inverse quantization and inverse transformation device 200 can apply the inverse adaptive color transformation device 100 described in the first embodiment as the inverse adaptive color transformation unit 203. In this case, the inverse adaptive color transformation unit 203 has a configuration similar to that of the inverse adaptive color transformation device 100 and performs similar processing.

[0267] For example, in the inverse adaptive color conversion unit 203, the cropping processing unit 102 crops the coefficient data res_x' supplied from the inverse orthogonal conversion unit 202 at a level based on the bit depth of the coefficient data. Then, the inverse YCgCo-R conversion unit 103 performs inverse YCgCo-R conversion on the coefficient data res_x' cropped at the level by the cropping processing unit 102 to obtain coefficient data res_x after inverse adaptive color conversion processing. Then, the inverse YCgCo-R conversion unit 103 outputs the obtained coefficient data res_x to the outside of the inverse quantization and inverse conversion device 200.

[0268] In this manner, the inverse adaptive color conversion unit 203 can clip the coefficient data res_x' to a level that does not increase distortion in the coefficient data res_x after inverse adaptive color conversion. Therefore, as in the inverse adaptive color conversion apparatus 100, the inverse adaptive color conversion unit 203 can suppress an increase in distortion in the coefficient data res_x after inverse adaptive color conversion while also suppressing an increase in the load of the inverse adaptive color conversion process. Therefore, the inverse quantization and inverse conversion apparatus 200 can suppress an increase in distortion in the coefficient data res_x to be output while also suppressing an increase in the load of the inverse quantization and inverse conversion process.

[0269] Then, as in the case of the inverse adaptive color conversion device 100, the inverse adaptive color conversion unit 203 can apply the various methods of the present technology described above in <1. Clipping Processing for Lossless Inverse Adaptive Color Conversion> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the inverse quantization and inverse conversion device 200 can apply the various methods of the present technology described above in <1. Clipping Processing for Lossless Inverse Adaptive Color Conversion>. By applying any of these methods, the inverse quantization and inverse conversion device 200 can achieve effects similar to those described in <Application of the Present Technology to the Inverse Adaptive Color Conversion Device>.

[0270] <Flow of Inverse Quantization and Inverse Transformation Processing>

[0271] Next, we will refer to Figure 5The flowchart of FIG. 1 describes an example of the flow of the inverse quantization and inverse transform process performed by the inverse quantization and inverse transform device 200 .

[0272] When the inverse quantization and inverse transform process starts, in step S201 , the inverse quantization unit 201 of the inverse quantization and inverse transform device 200 inversely quantizes the quantization coefficient qcoef_x corresponding to each component identifier cIdx=0, 1, and 2 included in the processing target TU to obtain an orthogonal transform coefficient coef_x.

[0273] In step S202 , the inverse orthogonal transform unit 202 refers to the transform information Tinfo corresponding to each component identifier, and performs an inverse orthogonal transform on the orthogonal transform coefficient coef_x to obtain coefficient data res_x′.

[0274] In step S203 , the inverse adaptive color conversion unit 203 performs inverse adaptive color conversion processing, performs inverse YCgCo-R conversion on the coefficient data res_x′, and obtains coefficient data res_x after the inverse adaptive color conversion processing.

[0275] When the process of step S203 is completed, the inverse quantization and inverse transformation process is completed.

[0276] <Application of this technology in inverse quantization and inverse transform processing>

[0277] In such an inverse quantization and inverse transform process, the present technology described above in <1. Clipping process of lossless inverse adaptive color transform> can be applied. Figure 3 The inverse adaptive color conversion process described in the flowchart of FIG. 1 is applied as the inverse adaptive color conversion process of step S203 .

[0278] In this manner, the inverse adaptive color conversion unit 203 can clip the coefficient data res_x' at a level that does not increase distortion in the coefficient data res_x after inverse adaptive color conversion. Therefore, the inverse adaptive color conversion unit 203 can suppress an increase in the load of the inverse adaptive color conversion process while suppressing an increase in distortion in the coefficient data res_x after inverse adaptive color conversion.

[0279] That is, by performing the inverse quantization and inverse transform process as described above, the inverse quantization and inverse transform device 200 can suppress an increase in the load of the inverse quantization and inverse transform process while suppressing an increase in distortion of the coefficient data res_x to be output.

[0280] Note that in the inverse quantization and inverse transform process (the inverse quantization and inverse transform process of the inverse adaptive color transform process (step S203)), various methods of the present technology described above in <1. Cropping Processing of Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3") can be applied. By applying any of these methods, effects similar to those described in <Application of the Present Technology to an Inverse Quantization and Inverse Transform Device> can be obtained (that is, effects similar to those described in <Application of the Present Technology to an Inverse Adaptive Color Transform Device>).

[0281] <4. Third embodiment>

[0282] <Image Decoding Device>

[0283] The above-described inverse quantization and inverse transformation apparatus 200 (inverse quantization and inverse transformation processing) can be applied to an image decoding apparatus. Figure 6 : is a block diagram showing an example of the configuration of an image decoding device as one aspect of an image processing device to which the present technology is applied. Figure 6 The image decoding device 400 shown is a device for decoding coded data of a moving image. For example, the image decoding device 400 decodes coded data of a moving image coded by a coding method such as VVC, AVC, or HEVC described in the above non-patent literature. For example, the image decoding device 400 can decode coded data of a moving image coded by an image coding device 500 ( Figure 8 ) generates the encoded data (bit stream).

[0284] Note that in Figure 6 In the figure, the main processing units, data flow, etc. are shown, and Figure 6 That is, in the image decoding apparatus 400, there may be Figure 6 The processing units shown as blocks in FIG, or there may be processing units not shown in FIG. Figure 6 The same applies to the other drawings describing the processing units and the like in the image decoding apparatus 400.

[0285] like Figure 6 As shown, the image decoding apparatus 400 includes a control unit 401, a storage buffer 411, a decoding unit 412, an inverse quantization and inverse transformation unit 413, a calculation unit 414, a loop filter unit 415, a reordering buffer 416, a frame memory 417, and a prediction unit 418. Note that the prediction unit 418 includes an intra-frame prediction unit and an inter-frame prediction unit (not shown).

[0286] <Control Unit>

[0287] The control unit 401 performs processing related to decoding control. For example, the control unit 401 obtains encoding parameters (such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and filter information Finfo) included in the bitstream via the decoding unit 412. Furthermore, the control unit 401 can estimate encoding parameters not included in the bitstream. Furthermore, the control unit 401 controls each processing unit (from the storage buffer 411 to the prediction unit 418) of the image decoding device 400 based on the obtained (or estimated) encoding parameters to control decoding.

[0288] For example, the control unit 401 provides header information Hinfo to the inverse quantization and inverse transformation unit 413, the prediction unit 418, and the loop filter unit 415. Furthermore, the control unit 401 provides prediction mode information Pinfo to the inverse quantization and inverse transformation unit 413 and the prediction unit 418. Furthermore, the control unit 401 provides transformation information Tinfo to the inverse quantization and inverse transformation unit 413. Furthermore, the control unit 401 provides residual information Rinfo to the decoding unit 412. Furthermore, the control unit 401 provides filter information Finfo to the loop filter unit 415.

[0289] Of course, the above example is an example, and the present invention is not limited to this example. For example, each encoding parameter can be provided to any processing unit. In addition, other information can be provided to any processing unit.

[0290] <Header Information Hinfo>

[0291] For example, the header information Hinfo may include information such as video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), picture header (PH) and slice header (SH).

[0292] For example, the header information Hinfo may include information defining the image size (e.g., parameters such as PicWidth indicating the horizontal width of the image and PicHeight indicating the vertical width of the image). Furthermore, the header information Hinfo may include information defining the bit depth (e.g., parameters such as bitDepthY indicating the bit depth of the luma component and bitDepthC indicating the bit depth of the chroma component). Furthermore, the header information Hinfo may include a parameter, ChromaArrayType, indicating the chroma array type. Furthermore, the header information Hinfo may include parameters such as MaxCUSize indicating the maximum value of the CU size and MinCUSize indicating the minimum value of the CU size. Furthermore, the header information Hinfo may include parameters such as MaxQTDepth indicating the maximum depth of quadtree partitioning (also referred to as quadtree partitioning) and MinQTDepth indicating the minimum depth of quadtree partitioning. Furthermore, the header information Hinfo may include parameters such as MaxBTDepth indicating the maximum depth of binary tree partitioning and MinBTDepth indicating the minimum depth of binary tree partitioning. Furthermore, the header information Hinfo may include parameters such as MaxTSSize indicating a maximum value of a transform skip block (also referred to as a maximum transform skip block size).

[0293] In addition, the header information Hinfo may include information such as an on / off flag (also called a valid flag) that defines each encoding tool. For example, the header information Hinfo may include an on / off flag related to an orthogonal transform process or a quantization process. Note that the on / off flag of the encoding tool may also be interpreted as a flag indicating whether the syntax related to the encoding tool is present in the encoded data. In addition, when the value of the on / off flag is 1 (true), it may indicate that the encoding tool is available, and when the value of the on / off flag is 0 (false), it may indicate that the encoding tool is not available. The interpretation of the flag value (true or false) may be reversed.

[0294] <Prediction mode information Pinfo>

[0295] The prediction mode information Pinfo may include, for example, parameters such as PBSize indicating the size of a processing target prediction block (PB) (prediction block size), and information such as intra prediction mode information IPinfo and motion prediction information MVinfo.

[0296] The intra prediction mode information IPinfo may include, for example, information such as prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W 1005, 7.3.8.5 coding unit syntax, and IntraPredModeY indicating a luma intra prediction mode derived from the syntax.

[0297] Furthermore, the intra prediction mode information IPinfo may include, for example, information such as ccp_flag (cclmp_flag), mclm_flag, chroma_sample_loc_type_idx, chroma_mpm_idx, and IntraPredModeC indicating a luma intra prediction mode derived from these syntaxes.

[0298] ccp_flag (cclmp_flag) is an inter-component prediction flag and is flag information indicating whether inter-component linear prediction is applied. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0299] mclm_flag is a multi-class linear prediction mode flag and is information about the linear prediction mode (linear prediction mode information). More specifically, mclm_flag is flag information indicating whether the multi-class linear prediction mode is set. For example, when mclm_flag==0, it indicates a one-class mode (single-class mode) (e.g., CCLMP), and when mclm_flag==1, it indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0300] chroma_sample_loc_type_idx is a chroma sample location type identifier and is an identifier for identifying the type of pixel location of the color difference component (also referred to as the chroma sample location type). Note that this chroma sample location type identifier (chroma_sample_loc_type_idx) is transmitted as chroma_sample_loc_info() (that is, the chroma sample location type identifier is stored in chroma_sample_loc_info()). This chroma_sample_loc_info() is information about the pixel location of the color difference component.

[0301] chroma_mpm_idx is a chroma MPM identifier, and is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is designated as the chroma intra prediction mode.

[0302] The motion prediction information MVinfo may include, for example, information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0, 1}, and mvd (see, for example, JCTVC-W 1005, 7.3.8.6 Prediction Unit Syntax).

[0303] Of course, the information included in the prediction mode information Pinfo is arbitrary, and information other than these information may be included in the prediction mode information Pinfo.

[0304] <Transformation information Tinfo>

[0305] The transform information Tinfo may include, for example, information such as TBWSize, TBHSize, ts_flag, scanldx, a quantization parameter qP, and a quantization matrix scaling_matrix (see, for example, JCTVC-W 1005, 7.3.4 Scaling List Data Syntax).

[0306] TBWSize is a parameter indicating the horizontal width size of the transform block to be processed. Note that the transform information Tinfo may include a value log2TBWSize, which is the logarithmic value of TBWSize with a base of 2 (instead of TBWSize). TBHSize is a parameter indicating the vertical width size of the transform block to be processed. Note that the transform information Tinfo may include a value log2TBHSize, which is the logarithmic value of TBHSize with a base of 2 (instead of TBHSize).

[0307] ts_flag is a transform skip flag, and is flag information indicating whether to skip (inverse) primary transform and (inverse) secondary transform. scanIdx is a scan identifier.

[0308] Of course, the information included in the transformation information Tinfo is arbitrary, and information other than these information may be included in the transformation information Tinfo.

[0309] <Residual information Rinfo>

[0310] The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding Syntax of JCTVC-W 1005) may include, for example, the following information.

[0311] cbf(coded_block_flag): residual data presence / absence flag

[0312] last_sig_coeff_x_pos: the last non-zero coefficient X coordinate

[0313] last_sig_coeff_y_pos: the last non-zero coefficient Y coordinate

[0314] coded_sub_block_flag: Sub-block non-zero coefficient presence / absence flag

[0315] sig_coeff_flag: non-zero coefficient presence / absence flag

[0316] gr1_flag: A flag indicating whether the level of the non-zero coefficient is greater than 1 (also called the GR1 flag) gr2_flag: A flag indicating whether the level of the non-zero coefficient is greater than 2 (also called the GR2 flag) sign_flag: A sign indicating the positive or negative sign of the non-zero coefficient (also called the sign code)

[0317] coeff_abs_level_remaining: The residual level of non-zero coefficients (also called the non-zero coefficient residual level)

[0318] Of course, the information included in the residual information Rinfo is arbitrary and may include information other than these information.

[0319] <Filter information Finfo>

[0320] The filter information Finfo may include, for example, control information related to each filter process described below.

[0321] Control information about the deblocking filter (DBF)

[0322] Control information about pixel adaptive offset (SAO)

[0323] Control information about the adaptive loop filter (ALF)

[0324] Control information about other linear / nonlinear filters

[0325] In addition, for example, the picture to which each filter is applied, information specifying an area in the picture, on / off control information of the filter in units of CUs, on / off control information of the filter related to the boundary between slices and tiles, etc. may be included. Of course, the information included in the filter information Finfo is arbitrary, and information other than this information may be included.

[0326] <Storage Buffer>

[0327] The storage buffer 411 acquires and holds (stores) a bit stream input to the image decoding device 400. The storage buffer 411 extracts encoded data included in the accumulated bit stream at predetermined timing or when a predetermined condition is satisfied, and supplies the encoded data to the decoding unit 412.

[0328] <Decoding Unit>

[0329] The decoding unit 412 acquires the encoded data supplied from the storage buffer 411 , performs entropy decoding (lossless decoding) on ​​the syntax value of each syntax element from the bit string according to the definition of the syntax table, and derives encoding parameters.

[0330] The encoding parameters may include, for example, information such as header information Hinfo, prediction mode information Pinfo, transform information Tinfo, residual information Rinfo, and filter information Finfo. That is, the decoding unit 412 decodes and parses (analyzes and obtains) this information from the bitstream.

[0331] The decoding unit 412 performs such processing (decoding, parsing, etc.) under the control of the control unit 401 , and supplies obtained information to the control unit 401 .

[0332] Furthermore, the decoding unit 412 decodes the coded data with reference to the residual information Rinfo. At this time, for example, the decoding unit 412 applies entropy decoding (lossless decoding) such as CABAC or CAVLC. That is, the decoding unit 412 decodes the coded data using a decoding method corresponding to the encoding method of the encoding process performed by the encoding unit 514 of the image encoding device 500.

[0333] For example, assuming CABAC is applied, the decoding unit 412 performs arithmetic decoding on the encoded data using the context model and derives the quantization coefficient level at each coefficient position in each transform block. The decoding unit 412 provides the derived quantization coefficient level to the inverse quantization and inverse transform unit 413.

[0334] <Inverse quantization and inverse transformation unit>

[0335] The inverse quantization and inverse transformation unit 413 acquires the quantization coefficient level supplied from the decoding unit 412. The inverse quantization and inverse transformation unit 413 acquires encoding parameters such as the prediction mode information Pinfo and the transformation information Tinfo supplied from the control unit 401.

[0336] The inverse quantization and inverse transformation unit 413 performs inverse quantization and inverse transformation processing on the quantization coefficient level based on the encoding parameters such as the prediction mode information Pinfo and the transformation information Tinfo to obtain residual data D'. The inverse quantization and inverse transformation processing is performed by the transformation quantization unit 513 ( Figure 8 ) is an inverse process of the transform and quantization process performed in the inverse quantization and inverse transform process. That is, for example, in the inverse quantization and inverse transform process, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are performed. Inverse quantization is the inverse process of quantization performed in the transform and quantization unit 513. Inverse orthogonal transform is the inverse process of orthogonal transform performed in the transform and quantization unit 513. Inverse adaptive color transform is the inverse process of adaptive color transform performed in the transform and quantization unit 513. Of course, the processes included in the inverse quantization and inverse transform process are arbitrary, and part of the above processes may be omitted, or processes other than the above processes may be included. The inverse quantization and inverse transform unit 413 provides the obtained residual data D' to the calculation unit 414.

[0337] <Computational Unit>

[0338] The calculation unit 414 receives the residual data D' provided by the inverse quantization and inverse transformation unit 413 and the predicted image provided by the prediction unit 418. The calculation unit 414 adds the residual data D' and the predicted image P corresponding to the residual data D' to obtain a local decoded image. The calculation unit 414 provides the obtained local decoded image to the loop filter unit 415 and the frame memory 417.

[0339] <Loop filter unit>

[0340] The loop filter unit 415 acquires the local decoded image supplied from the calculation unit 414. The loop filter unit 415 acquires the filter information Finfo supplied from the control unit 401. Note that information input to the loop filter unit 415 is arbitrary, and information other than these information may be input.

[0341] The loop filter unit 415 appropriately performs filtering processing on the local decoded image based on the filter information Finfo. For example, the loop filter unit 415 can apply a bilateral filter as filtering processing. For example, the loop filter unit 415 can apply a deblocking filter (DBF) as filtering processing. For example, the loop filter unit 415 can apply an adaptive offset filter (sample adaptive offset (SAO)) as filtering processing. For example, the loop filter unit 415 can apply an adaptive loop filter (ALF) as filtering processing. In addition, the loop filter unit 415 can apply multiple filters in combination as filtering processing. Note that which filter is applied and in what order the filters are applied are arbitrary and can be appropriately selected. For example, the loop filter unit 415 applies the following four loop filters in the following order as filtering processing: bilateral filter, deblocking filter, adaptive offset filter and adaptive loop filter.

[0342] The loop filter unit 415 performs a filtering process corresponding to the filtering process performed by the encoding side device. For example, the loop filter unit 415 performs a filtering process corresponding to the filtering process performed by the loop filter unit 518 ( Figure 8 ) performs filtering processing corresponding to the filtering processing performed by the loop filter unit 415. Of course, the filtering processing performed by the loop filter unit 415 is arbitrary and is not limited to the above example. For example, the loop filter unit 415 can apply a Wiener filter or the like.

[0343] The loop filter unit 415 supplies the local decoded image subjected to the filtering process to the reordering buffer 416 and the frame memory 417 .

[0344] <Reorder Buffer>

[0345] The reordering buffer 416 receives the local decoded images provided by the loop filter unit 415 as input and holds (stores) the local decoded images. The reordering buffer 416 reconstructs a decoded image for each picture unit using the local decoded images and holds (stores) the decoded images in the buffer. The reordering buffer 416 reorders the decoded images obtained from the decoding order into the reproduction order. The reordering buffer 416 outputs the decoded image group reordered in the reproduction order as motion image data to the outside of the image decoding device 400.

[0346] <Frame Memory>

[0347] The frame memory 417 acquires the local decoded image supplied from the calculation unit 414, reconstructs the decoded image for each picture unit, and stores the decoded image in the buffer of the frame memory 417. Furthermore, the frame memory 417 acquires the local decoded image subjected to the loop filtering process supplied from the loop filter unit 415, reconstructs the decoded image for each picture unit, and stores the decoded image in the buffer of the frame memory 417.

[0348] The frame memory 417 appropriately provides the stored decoded image (or a portion thereof) to the prediction unit 418 as a reference image. Note that the frame memory 417 may store header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, and the like related to the generation of the decoded image.

[0349] <Prediction Unit>

[0350] The prediction unit 418 obtains the prediction mode information Pinfo supplied from the control unit 401. Furthermore, the prediction unit 418 obtains the decoded image (or a portion thereof) read from the frame memory 417. The prediction unit 418 performs prediction processing in the prediction mode employed during encoding based on the prediction mode information Pinfo, and generates a predicted image P by referencing the decoded image as a reference image. The prediction unit 418 supplies the generated predicted image P to the calculation unit 414.

[0351] <Application of this technology in image decoding device>

[0352] In such an image decoding device 400, the present technology described above in <1. Cropping process of lossless inverse adaptive color transform> can be applied. That is, the image decoding device 400 can apply the inverse quantization and inverse transformation device 200 described in the second embodiment as the inverse quantization and inverse transformation unit 413. In this case, the inverse quantization and inverse transformation unit 413 has the same functions as the inverse quantization and inverse transformation device 200 ( Figure 4 ) and performs similar processing.

[0353] For example, in the inverse quantization and inverse transformation unit 413, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the decoding unit 412 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 inversely quantizes the quantization coefficient qcoef_x by using information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag to obtain the orthogonal transformation coefficient coef_x.

[0354] The inverse orthogonal transform unit 202 performs inverse orthogonal transform on the orthogonal transform coefficient coef_x obtained by the inverse quantization unit 201 using information such as transform information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx, and obtains coefficient data res_x′.

[0355] The inverse adaptive color transform unit 203 appropriately performs inverse adaptive color transform (inverse YCgCo-R transform) on the coefficient data res_x′ obtained by the inverse orthogonal transform unit 202 using a lossless method based on the cu_act_enabled_flag, thereby obtaining coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the obtained coefficient data res_x as residual data D′ to the calculation unit 414.

[0356] As described in the first embodiment, the inverse adaptive color conversion unit 203 has a configuration similar to that of the inverse adaptive color conversion device 100 and performs similar processing. For example, in the inverse adaptive color conversion unit 203, the cropping processing unit 102 crops the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. The inverse YCgCo-R conversion unit 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' cropped at the level by the cropping processing unit 102 to obtain coefficient data res_x after the inverse adaptive color conversion process. The inverse YCgCo-R conversion unit 103 then supplies the obtained coefficient data res_x as residual data D' to the calculation unit 414.

[0357] In this manner, the inverse quantization and inverse transformation unit 413 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color transformation. Therefore, similar to the inverse quantization and inverse transformation device 200, the inverse quantization and inverse transformation unit 413 can suppress an increase in the load of the inverse quantization and inverse transformation process while suppressing an increase in the distortion of the coefficient data res_x (residual data D') to be output. In other words, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0358] Then, as in the case of the inverse quantization and inverse transformation device 200, the inverse quantization and inverse transformation unit 413 can apply the various methods of the present technology described above in <1. Cropping Process for Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the image decoding device 400 can apply the various methods of the present technology described above in <1. Cropping Process for Lossless Inverse Adaptive Color Transform>. By applying any of these methods, the image decoding device 400 can obtain an effect similar to that described in <Application of the Present Technology to the Inverse Quantization and Inverse Transform Device> (that is, an effect similar to that described in <Application of the Present Technology to the Inverse Adaptive Color Transform Device>).

[0359] <Image Decoding Processing Flow>

[0360] Next, the flow of each process performed by the image decoding device 400 as described above will be described. Figure 7 The flowchart describes an example of the flow of image decoding processing.

[0361] When the image decoding process starts, in step S401 , the storage buffer 411 acquires and holds (stores) a bit stream (encoded data) supplied from the outside of the image decoding device 400 .

[0362] In step S402, the decoding unit 412 performs decoding processing. For example, the decoding unit 412 parses (analyzes and obtains) various encoding parameters (e.g., header information Hinfo, prediction mode information Pinfo, transform information Tinfo, etc.) from the bitstream. The control unit 401 provides the obtained various encoding parameters to various processing units to set the various encoding parameters.

[0363] Furthermore, the control unit 401 sets the processing unit based on the obtained encoding parameter. Furthermore, the decoding unit 412 decodes the bit stream according to the control of the control unit 401 and derives the quantization coefficient level.

[0364] In step S403 , the inverse quantization and inverse transformation unit 413 performs an inverse quantization and inverse transformation process to obtain residual data D′.

[0365] In step S404 , the prediction unit 418 generates a predicted image P. For example, the prediction unit 418 performs prediction processing by a prediction method specified by the encoding side based on the encoding parameters set in step S402 and the like, and generates the predicted image P by referring to a reference image stored in the frame memory 417 .

[0366] In step S405 , the calculation unit 414 adds the residual data D′ obtained in step S403 to the predicted image P obtained in step S404 to obtain a local decoded image.

[0367] In step S406 , the loop filter unit 415 performs a loop filtering process on the local decoded image obtained by the process in step S405 .

[0368] In step S407, the reordering buffer 416 uses the local decoded images filtered by the process in step S406 to generate decoded images and reorders the decoded image groups from the decoding order to the reproduction order. The decoded image groups reordered in the reproduction order are output as moving images to the outside of the image decoding device 400.

[0369] In addition, in step S408 , the frame memory 417 stores at least one of the local decoded image obtained by the process in step S405 or the local decoded image filtered by the process in step S406 .

[0370] When the process of step S408 ends, the image decoding process ends.

[0371] <Application of this technology in image decoding processing>

[0372] In such an image decoding process, the present technology described above in <1. Cropping process of lossless inverse adaptive color transform> can be applied. Figure 5 The inverse quantization and inverse transform processing described in the flowchart of FIG. 4 is applied as the inverse quantization and inverse transform processing of step S403 .

[0373] In this way, the inverse quantization and inverse transformation unit 413 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation. Therefore, the inverse quantization and inverse transformation unit 413 can suppress an increase in the load of the inverse quantization and inverse transformation process while suppressing an increase in the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation.

[0374] That is, by performing the image decoding process as described above, the image decoding device 400 can suppress an increase in the load of the image decoding process while suppressing a decrease in the quality of the decoded image.

[0375] Note that in the image decoding process (inverse quantization and inverse transformation process (step S403)), various methods of the present technology described above in <1. Cropping process of lossless inverse adaptive color transformation> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, an effect similar to that described in <Application of the present technology to an image decoding device> can be obtained (that is, an effect similar to that described in <Application of the present technology to an inverse adaptive color transformation device>).

[0376] <5. Fourth embodiment>

[0377] <Image Coding Device>

[0378] The above-described inverse quantization and inverse transformation device 200 (inverse quantization and inverse transformation process) can be applied to an image encoding device. Figure 8 : is a block diagram showing an example of the configuration of an image encoding device as one aspect of an image processing device to which the present technology is applied. Figure 8 The image encoding device 500 shown is a device for encoding image data of a moving image. For example, the image encoding device 500 encodes the image data of the moving image by using a coding method such as Versatile Video Coding (VVC), Advanced Video Coding (AVC), or High Efficiency Video Coding (HEVC) described in the above-mentioned non-patent literature. For example, the image encoding device 500 can generate a video image that can be decoded by the above-mentioned image decoding device 400 ( Figure 6 )Decoded encoded data (bit stream).

[0379] Note that in Figure 8 In the figure, the main processing units, data flow, etc. are shown, and Figure 8 That is, in the image coding apparatus 500, there may be Figure 8 The processing units shown as blocks in FIG, or there may be processing units not shown in FIG. Figure 8 The same applies to the other drawings describing the processing units and the like in the image encoding device 500.

[0380] like Figure 8 As shown, the image encoding device 500 includes a control unit 501, a reordering buffer 511, a calculation unit 512, a transform and quantization unit 513, an encoding unit 514, and a storage buffer 515. In addition, the image encoding device 500 includes an inverse quantization and inverse transform unit 516, a calculation unit 517, a loop filter unit 518, a frame memory 519, a prediction unit 520, and a rate control unit 521.

[0381] <Control Unit>

[0382] The control unit 501 divides the moving image data held by the reordering buffer 511 into blocks of processing units (CU, PU, ​​TU, etc.) based on the block size of the processing unit specified externally or in advance. Furthermore, the control unit 501 determines the encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be provided to each block based on, for example, rate-distortion optimization (RDO). For example, the control unit 501 may set a transform skip flag, etc.

[0383] After determining the encoding parameters described above, the control unit 501 provides the encoding parameters to each block. For example, header information Hinfo is provided to each block. Prediction mode information Pinfo is provided to the encoding unit 514 and the prediction unit 520. Transform information Tinfo is provided to the encoding unit 514, the transform and quantization unit 513, and the inverse quantization and inverse transform unit 516. Filter information Finfo is provided to the encoding unit 514 and the loop filter unit 518. Of course, the destination for providing each encoding parameter is arbitrary and is not limited to this example.

[0384] <Reorder Buffer>

[0385] Each field of moving image data (input image) is input to the image encoding device 500 in the order of reproduction (display order). The reordering buffer 511 acquires and holds (stores) each input image in its order of reproduction (display order). Under the control of the control unit 501, the reordering buffer 511 reorders the input image in the order of encoding (decoding order) or divides the input image into blocks in the processing unit. The reordering buffer 511 supplies each processed input image to the computing unit 512.

[0386] <Computational Unit>

[0387] The calculation unit 512 subtracts the predicted image P supplied from the prediction unit 520 from the image corresponding to the block in the processing unit supplied from the rearrangement buffer 511 to obtain residual data D, and supplies the residual data D to the transform and quantization unit 513 .

[0388] <Transform Quantization Unit>

[0389] The transform and quantization unit 513 receives the residual data D provided by the calculation unit 512. Furthermore, the transform and quantization unit 513 receives the prediction mode information Pinfo and the transform information Tinfo provided by the control unit 501. Based on the prediction mode information Pinfo and the transform information Tinfo, the transform and quantization unit 513 performs transform and quantization processing on the residual data D to obtain quantization coefficient levels. The transform and quantization processing includes, for example, adaptive color transform, orthogonal transform, and quantization. Of course, the processing included in the transform and quantization processing is arbitrary, and some of the aforementioned processing may be omitted, or processing other than the aforementioned processing may be included. The transform and quantization unit 513 provides the obtained quantization coefficient levels to the encoding unit 514 and the inverse quantization and inverse transform unit 516.

[0390] <Coding unit>

[0391] The encoding unit 514 obtains the quantization coefficient level provided by the transform and quantization unit 513. Furthermore, the encoding unit 514 obtains various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) provided by the control unit 501. Furthermore, the encoding unit 514 obtains filter information (e.g., filter coefficients) provided by the loop filter unit 518. Furthermore, the encoding unit 514 obtains information on the optimal prediction mode provided by the prediction unit 520.

[0392] The encoding unit 514 performs entropy encoding (lossless encoding) on ​​the quantized coefficient levels to generate a bit string (encoded data). The encoding unit 514 can apply, for example, context-adaptive binary arithmetic coding (CABAC) as entropy encoding. The encoding unit 514 can also apply, for example, context-adaptive variable length coding (CAVLC) as entropy encoding. Of course, the content of this entropy encoding is arbitrary and is not limited to these examples.

[0393] Furthermore, the encoding unit 514 derives residual information Rinfo from the quantization coefficient level, encodes the residual information Rinfo, and generates a bit string.

[0394] Furthermore, the encoding unit 514 includes information about the filter supplied from the loop filter unit 518 in the filter information Finfo, and includes information about the optimal prediction mode supplied from the prediction unit 520 in the prediction mode information Pinfo. The encoding unit 514 then encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0395] Furthermore, the encoding unit 514 multiplexes the bit strings of the various types of information generated as described above to generate encoded data, and supplies the encoded data to the storage buffer 515 .

[0396] <Storage Buffer>

[0397] The storage buffer 515 temporarily stores the encoded data obtained by the encoding unit 514. The storage buffer 515 outputs the stored encoded data as, for example, a bit stream to the outside of the image encoding device 500 at a predetermined timing. For example, the encoded data is transmitted to the decoding side via any recording medium, any transmission medium, any information processing device, etc. In other words, the storage buffer 515 also serves as a transmission unit for transmitting the encoded data (bit stream).

[0398] <Inverse quantization and inverse transformation unit>

[0399] The inverse quantization and inverse transformation unit 516 acquires the quantization coefficient level supplied from the transform and quantization unit 513. Furthermore, the inverse quantization and inverse transformation unit 516 acquires the transform information Tinfo supplied from the control unit 501.

[0400] The inverse quantization and inverse transformation unit 516 performs inverse quantization and inverse transformation processing on the quantization coefficient level based on the transformation information Tinfo to obtain residual data D'. This inverse quantization and inverse transformation processing is the inverse of the transform and quantization processing performed in the transform and quantization unit 513, and is similar to the inverse quantization and inverse transformation processing performed in the inverse quantization and inverse transformation unit 413 of the image decoding device 400 described above.

[0401] That is, in the inverse quantization and inverse transform process, for example, processes such as inverse quantization, inverse orthogonal transform, and inverse adaptive color transform are performed. This inverse quantization is the inverse process of quantization performed in the transform and quantization unit 513, and is similar to the inverse quantization performed in the inverse quantization and inverse transform unit 413. In addition, the inverse orthogonal transform is the inverse process of the orthogonal transform performed in the transform and quantization unit 513, and is similar to the inverse orthogonal transform performed in the transform and quantization unit 513. In addition, the inverse adaptive color transform is the inverse process of the adaptive color transform performed in the transform and quantization unit 513, and is similar to the inverse adaptive color transform performed in the transform and quantization unit 513.

[0402] Of course, the processes included in the inverse quantization and inverse transformation process are arbitrary, and some of the above processes may be omitted, or processes other than the above processes may be included. The inverse quantization and inverse transformation unit 516 supplies the obtained residual data D′ to the calculation unit 517 .

[0403] <Computational Unit>

[0404] The calculation unit 517 obtains the residual data D' provided by the inverse quantization and inverse transformation unit 516 and the predicted image P provided by the prediction unit 520. The calculation unit 517 adds the residual data D' and the predicted image P corresponding to the residual data D' to obtain a local decoded image. The calculation unit 517 provides the obtained local decoded image to the loop filter unit 518 and the frame memory 519.

[0405] <Loop filter unit>

[0406] The loop filter unit 518 obtains the local decoded image provided by the calculation unit 517. In addition, the loop filter unit 518 obtains the filter information Finfo provided by the control unit 501. In addition, the loop filter unit 518 obtains the input image (original image) provided by the reordering buffer 511. Note that the information input to the loop filter unit 518 is arbitrary, and information other than this information can be input. For example, the prediction mode, motion information, code amount target value, quantization parameter qP, picture type, block (CU, CTU, etc.) information, etc. can be input to the loop filter unit 518 as needed.

[0407] The loop filter unit 518 appropriately performs a filtering process on the local decoded image based on the filter information Finfo. The loop filter unit 518 also uses an input image (original image) and other input information for the filtering process as necessary.

[0408] For example, the loop filter unit 518 can apply a bilateral filter as a filtering process. For example, the loop filter unit 518 can apply a deblocking filter (DBF) as a filtering process. For example, the loop filter unit 518 can apply an adaptive offset filter (sample adaptive offset (SAO)) as a filtering process. For example, the loop filter unit 518 can apply an adaptive loop filter (ALF) as a filtering process. In addition, the loop filter unit 518 can apply a plurality of filters in combination as a filtering process. Note that which filter is applied and in what order the filters are applied are arbitrary and can be appropriately selected. For example, the loop filter unit 518 applies the following four loop filters in the following order as filtering processes: a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter.

[0409] Of course, the filtering process performed by the loop filter unit 518 is arbitrary and not limited to the above example. For example, the loop filter unit 518 can apply a Wiener filter or the like.

[0410] The loop filter unit 518 supplies the filtered local decoded image to the frame memory 519. Note that, for example, in the case where information about the filter such as the filter coefficient is transmitted to the decoding side, the loop filter unit 518 supplies the information about the filter to the encoding unit 514.

[0411] <Frame Memory>

[0412] The frame memory 519 performs processing related to the storage of image-related data. For example, the frame memory 519 obtains the local decoded image provided by the calculation unit 517 and the local decoded image subjected to filtering provided by the loop filter unit 518, and holds (stores) the local decoded image. In addition, the frame memory 519 uses the local decoded image to reconstruct and hold the decoded image for each picture unit (storing the decoded image in the buffer of the frame memory 519). In response to a request from the prediction unit 520, the frame memory 519 supplies the decoded image (or a portion thereof) to the prediction unit 520.

[0413] <Prediction Unit>

[0414] The prediction unit 520 performs processing related to the generation of a predicted image. For example, the prediction unit 520 obtains the prediction mode information Pinfo provided by the control unit 501. For example, the prediction unit 520 obtains the input image (original image) provided by the reordering buffer 511. For example, the prediction unit 520 obtains the decoded image (or a portion thereof) read from the frame memory 519.

[0415] The prediction unit 520 uses the prediction mode information Pinfo and the input image (original image) to perform prediction processing such as inter-frame prediction or intra-frame prediction. That is, the prediction unit 520 generates a predicted image P by performing prediction and motion compensation with reference to the decoded image as a reference image. The prediction unit 520 provides the generated predicted image P to the calculation unit 512 and the calculation unit 517. In addition, the prediction unit 520 provides information about the prediction mode selected by the above process (i.e., the optimal prediction mode) to the encoding unit 514 as needed.

[0416] <Rate Control Unit>

[0417] The rate control unit 521 performs processing related to rate control. For example, the rate control unit 521 controls the rate of quantization operation of the transform and quantization unit 513 based on the code amount of the encoded data accumulated in the storage buffer 515 so that overflow or underflow does not occur.

[0418] <Transform Quantization Unit>

[0419] Figure 9 It shows Figure 8This is a block diagram of a main configuration example of the transform and quantization unit 513. Figure 9 As shown, the transform and quantization unit 513 includes an adaptive color transform unit 541 , an orthogonal transform unit 542 , and a quantization unit 543 .

[0420] The adaptive color conversion unit 541 obtains the residual data D( Figure 8 ) as coefficient data res_x. Furthermore, the adaptive color transform unit 541 acquires the cu_act_enabled_flag supplied from the control unit 501. The adaptive color transform unit 541 performs adaptive color transform on the coefficient data res_x based on the value of the cu_act_enabled_flag. For example, when the cu_act_enabled_flag is true (e.g., "1"), the adaptive color transform unit 541 performs lossless adaptive color transform (YCgCo-R transform) and transforms the coefficient data res_x in the RGB domain into coefficient data res_x' in the YCgCo domain. Adaptive color transform is the inverse process of the inverse adaptive color transform performed in the inverse quantization and inverse transform unit 413 or the inverse quantization and inverse transform unit 516. The adaptive color transform unit 541 supplies the resulting coefficient data res_x' to the orthogonal transform unit 542.

[0421] The orthogonal transform unit 542 obtains the coefficient data res_x' supplied from the adaptive color transform unit 541. The orthogonal transform unit 542 obtains information such as transform information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501. The orthogonal transform unit 542 uses the obtained information to orthogonally transform the coefficient data res_x' to obtain orthogonal transform coefficients coef_x. This orthogonal transform is the inverse of the inverse orthogonal transform performed in the inverse quantization and inverse transform unit 413 or the inverse quantization and inverse transform unit 516. The orthogonal transform unit 542 supplies the obtained orthogonal transform coefficients coef_x to the quantization unit 543.

[0422] The quantization unit 543 acquires the orthogonal transform coefficient coeff_x supplied from the orthogonal transform unit 542. In addition, the quantization unit 543 acquires information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501. The quantization unit 543 quantizes the orthogonal transform coefficient coef_x using the acquired information and derives a quantized coefficient qcoef_x. Quantization is an inverse process of the inverse quantization performed in the inverse quantization inverse transform unit 413 or the inverse quantization inverse transform unit 516. The quantization unit 543 supplies the derived quantized coefficient qcoef_x to the encoding unit 514 and the inverse quantization inverse transform unit 516 ( Figure 8 ) as the quantization coefficient level.

[0423] <Adaptive Color Conversion Unit>

[0424] Figure 10 It shows Figure 9 A block diagram showing an example of the main configuration of the adaptive color conversion unit 541 in FIG. Figure 10 As shown, the adaptive color conversion unit 541 includes a selection unit 571 and a YCgCo-R conversion unit 572.

[0425] The selection unit 571 obtains the residual data D( Figure 8 ) as the coefficient data res_x. In addition, the selection unit 571 obtains the cu_act_enabled_flag provided from the control unit 501. The selection unit 571 selects whether to perform adaptive color transform on the obtained coefficient data res_x based on the value of cu_act_enabled_flag. For example, when cu_act_enabled_flag is true (e.g., "1"), the selection unit 571 determines that adaptive color transform is applicable and provides the coefficient data res_x to the YCgCo-R conversion unit 572.

[0426] For example, when cu_act_enabled_flag is false (e.g., “0”), the selection unit 571 determines that application of the adaptive color transform (inverse adaptive color transform) is prohibited (that is, not applicable), and supplies the coefficient data res_x as coefficient data res_x′ after application of the color transform to the orthogonal transform unit 542 ( Figure 9 ).

[0427] The YCgCo-R conversion unit 572 acquires the coefficient data res_x supplied from the selection unit 101. The YCgCo-R conversion unit 572 performs a YCgCo-R conversion on the acquired coefficient data res_x to obtain coefficient data res_x′ subjected to the YCgCo-R conversion. This YCgCo-R conversion is an inverse process of the inverse YCgCo-R conversion performed in the inverse quantization inverse conversion unit 413 or the inverse quantization inverse conversion unit 516. The YCgCo-R conversion unit 572 supplies the obtained coefficient data res_x′ to the orthogonal conversion unit 542 ( Figure 9 ).

[0428] <Application of this technology in image encoding device>

[0429] In such an image encoding device 500 ( Figure 8), the present technology described above in <1. Cropping process of lossless inverse adaptive color transform> can be applied. That is, the image encoding device 500 can apply the inverse quantization and inverse transformation device 200 described in the second embodiment as the inverse quantization and inverse transformation unit 516. In this case, the inverse quantization and inverse transformation unit 516 has the same functions as the inverse quantization and inverse transformation device 200 ( Figure 4 ) is configured similarly and performs similar processing.

[0430] For example, in the inverse quantization and inverse transformation unit 516, the inverse quantization unit 201 obtains the quantization coefficient level supplied from the transform and quantization unit 513 as the quantization coefficient qcoef_x. Then, the inverse quantization unit 201 inversely quantizes the quantization coefficient qcoef_x by using information such as the quantization parameter qP, cu_act_enabled_flag, and transform_skip_flag supplied from the control unit 501, and obtains the orthogonal transform coefficient coef_x.

[0431] The inverse orthogonal transform unit 202 inversely orthogonally transforms the orthogonal transform coefficient coef_x derived by the inverse quantization unit 201 using information such as transform information Tinfo, transform_skip_flag, mts_idx, and lfnst_idx supplied from the control unit 501 , and derives coefficient data res_x′.

[0432] Based on the cu_act_enabled_flag supplied from the control unit 501, the inverse adaptive color transform unit 203 appropriately performs inverse adaptive color transform (inverse YCgCo-R transform) on the coefficient data res_x′ obtained by the inverse orthogonal transform unit 202 using a lossless method to obtain coefficient data res_x after the inverse adaptive color transform process. The inverse adaptive color transform unit 203 supplies the obtained coefficient data res_x as residual data D′ to the calculation unit 517.

[0433] As described in the first embodiment, the inverse adaptive color conversion unit 203 has a configuration similar to that of the inverse adaptive color conversion device 100 and performs similar processing. For example, in the inverse adaptive color conversion unit 203, the cropping processing unit 102 crops the coefficient data res_x' supplied from the inverse orthogonal transform unit 202 at a level based on the bit depth of the coefficient data. The inverse YCgCo-R conversion unit 103 then performs inverse YCgCo-R conversion on the coefficient data res_x' cropped at the level by the cropping processing unit 102 to obtain coefficient data res_x after the inverse adaptive color conversion process. The inverse YCgCo-R conversion unit 103 then supplies the obtained coefficient data res_x as residual data D' to the calculation unit 517.

[0434] In this manner, the inverse quantization and inverse transformation unit 516 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x after the inverse adaptive color transformation. Therefore, similar to the inverse quantization and inverse transformation device 200, the inverse quantization and inverse transformation unit 516 can suppress an increase in the load of the inverse quantization and inverse transformation process while suppressing an increase in the distortion of the coefficient data res_x (residual data D') to be output. In other words, the image encoding device 500 can suppress an increase in the load of the image encoding process while suppressing a decrease in the quality of the decoded image.

[0435] Then, as in the case of the inverse quantization and inverse transformation device 200, the inverse quantization and inverse transformation unit 516 can apply the various methods of the present technology described above in <1. Cropping Process for Lossless Inverse Adaptive Color Transform> (including "Method 1," "Method 1-1," "Method 1-2," "Method 2," "Method 2-1," "Method 2-2," and "Method 3"). That is, the image encoding device 500 can apply the various methods of the present technology described above in <1. Cropping Process for Lossless Inverse Adaptive Color Transform>. By applying any of these methods, the image encoding device 500 can achieve effects similar to those described in <Application of the Present Technology to the Inverse Quantization and Inverse Transform Device> (that is, effects similar to those described in <Inverse Adaptive Color Transform Device>).

[0436] <Image Coding Processing Flow>

[0437] Next, we will refer to Figure 11 An example of the flow of image encoding processing performed by the image encoding device 500 as described above is described with reference to the flowchart in .

[0438] When the image encoding process starts, in step S501 , the reordering buffer 511 is controlled by the control unit 501 to reorder the order of frames of input moving image data from the display order to the encoding order.

[0439] In step S502 , the control unit 501 sets processing units (divides into blocks) for the input image held by the rearrangement buffer 511 .

[0440] In step S503 , the control unit 501 sets encoding parameters (for example, header information Hinfo, prediction mode information Pinfo, transform information Tinfo, etc.) for the input image held by the reordering buffer 511 .

[0441] In step S504, the prediction unit 520 performs prediction processing in the optimal prediction mode and generates a predicted image, etc. For example, in this prediction processing, the prediction unit 520 performs intra prediction in the optimal intra prediction mode to generate a predicted image, etc., performs inter prediction in the optimal inter prediction mode to generate a predicted image, etc., and selects the optimal prediction mode from the predicted image based on a cost function value, etc.

[0442] In step S505, the calculation unit 512 calculates the difference between the input image and the predicted image of the optimal mode selected by the prediction process in step S504. In other words, the calculation unit 512 obtains residual data D between the input image and the predicted image. The residual data D obtained in this manner has a smaller data volume than the original image data. Therefore, the data volume can be compressed compared to the case where the image is encoded as is.

[0443] In step S506 , the transform and quantization unit 513 performs transform and quantization processing on the residual data D obtained by the processing in step S505 by using the encoding parameters such as the transform information Tinfo set in step S503 , and obtains a quantization coefficient level.

[0444] In step S507, the inverse quantization and inverse transformation unit 516 performs inverse quantization and inverse transformation processing on the quantization coefficient level obtained in step S506 by using the encoding parameters such as the transformation information Tinfo set in step S503, and obtains residual data D'. This inverse quantization and inverse transformation processing is the inverse processing of the transformation and quantization processing of step S506, and is the same as Figure 7 The inverse quantization and inverse transformation process of step S403 of the image decoding process is similar to the process.

[0445] In step S508 , the calculation unit 517 adds the predicted image generated by the prediction process in step S504 and the residual data D′ obtained by the inverse quantization and inverse transform process in step S507 , thereby generating a local decoded image.

[0446] In step S509 , the loop filter unit 518 performs loop filtering processing on the local decoded image obtained by the processing in step S508 .

[0447] In step S510 , the frame memory 519 stores the local decoded image obtained by the process in step S508 and the local decoded image filtered in step S509 .

[0448] In step S511, the encoding unit 514 encodes the quantization coefficient levels obtained by the transform and quantization processing in step S506 to obtain encoded data. Furthermore, at this time, the encoding unit 514 encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, and transform information Tinfo). Furthermore, the encoding unit 514 derives residual information RInfo from the quantization coefficient levels and encodes the residual information RInfo.

[0449] In step S512, the storage buffer 515 stores the encoded data obtained in this manner and outputs the encoded data as, for example, a bitstream to the outside of the image encoding device 500. This bitstream is transmitted to a decoding-side device (e.g., the image decoding device 400) via, for example, a transmission path or a recording medium. Furthermore, the rate control unit 521 controls the rate as needed. When the processing of step S512 is completed, the image encoding process ends.

[0450] <Transform and Quantization Processing Flow>

[0451] Next, we will refer to Figure 12 The flowchart is described in Figure 11 An example of the flow of the transform and quantization processing performed in step S506.

[0452] When the transform and quantization process starts, in step S541, the adaptive color transform unit 541 generates the color quantization result based on the result in step S503 ( Figure 11 ) performs adaptive color transform on the coefficient data res_x obtained by the process of step S505 to obtain coefficient data res_x'.

[0453] In step S542, the orthogonal transform unit 542 transforms the Figure 11 ) performs an orthogonal transform on the coefficient data res_x' obtained in step S541, and obtains an orthogonal transform coefficient coef_x.

[0454] In step S543, the quantization unit 543 performs the quantization by using the Figure 11 ) and the like, quantizes the orthogonal transformation coefficient coef_x obtained in step S542, and obtains the quantized coefficient qcoef_x.

[0455] When the processing of step S543 is completed, the processing returns to Figure 11 .

[0456] <Flow of Adaptive Color Conversion Processing>

[0457] Next, we will refer to Figure 13 The flowchart is described in Figure 12 An example of the flow of the adaptive color conversion process performed in step S541 of FIG.

[0458] When the adaptive color conversion process starts, in step S571, the selection unit 571 of the adaptive color conversion unit 541 determines whether cu_act_enabled_flag is true. In the case where it is determined that cu_act_enabled_flag is true, the process proceeds to step S572.

[0459] In step S572 , the YCgCo-R conversion unit 572 performs YCgCo-R conversion on the coefficient data res_x of the three components to obtain adaptive color converted coefficient data res_x′.

[0460] In step S573, the YCgCo-R conversion unit 572 supplies the resulting coefficient data res_x' to the orthogonal conversion unit 542. When the process of step S573 ends, the adaptive color conversion process ends.

[0461] Also, in a case where it is determined in step S571 that cu_act_enabled_flag is false, the process proceeds to step S574 .

[0462] In step S574, the selection unit 571 supplies the coefficient data res_x as the coefficient data res_x' after the adaptive color conversion to the orthogonal conversion unit 542. When the process of step S574 ends, the adaptive color conversion process ends.

[0463] <Application of this technology in image coding processing>

[0464] In such an image encoding process, the present technology described above in <1. Cropping process of lossless inverse adaptive color transform> can be applied. Figure 5 The inverse quantization and inverse transform processing described in the flowchart of FIG. 5 is applied as the inverse quantization and inverse transform processing of step S507 .

[0465] In this way, the inverse quantization and inverse transformation unit 516 can clip the coefficient data res_x' at a level that does not increase the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation. Therefore, the inverse quantization and inverse transformation unit 516 can suppress an increase in the load of the inverse quantization and inverse transformation process while suppressing an increase in the distortion of the coefficient data res_x (residual data D') after the inverse adaptive color transformation.

[0466] That is, by performing the image encoding process as described above, the image encoding device 500 can suppress an increase in the load of the image encoding process while suppressing a decrease in the quality of the decoded image.

[0467] Note that in the image encoding process (inverse quantization and inverse transform process (step S507)), various methods of the present technology described above in <1. Cropping process of lossless inverse adaptive color transform> (including "Method 1", "Method 1-1", "Method 1-2", "Method 2", "Method 2-1", "Method 2-2", and "Method 3") can be applied. By applying any of these methods, an effect similar to that described in <Application of the present technology to an image encoding device> can be obtained (that is, an effect similar to that described in <Application of the present technology to an inverse adaptive color transform device>).

[0468] <6. Appendix>

[0469] <Computer>

[0470] The above series of processes can be performed by hardware or software. In the case of performing a series of processes by software, the program constituting the software is installed in a computer. Here, the computer includes, for example, a computer incorporated in dedicated hardware, a general-purpose personal computer capable of performing various functions by installing various programs, etc.

[0471] Figure 14 : is a block diagram showing a configuration example of hardware of a computer that executes the above-described series of processes by a program.

[0472] exist Figure 14 In the illustrated computer 800 , a central processing unit (CPU) 801 , a read-only memory (ROM) 802 , and a random access memory (RAM) 803 are connected to one another via a bus 804 .

[0473] An input / output interface 810 is also connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.

[0474] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, and input terminals. The output unit 812 includes, for example, a display, a speaker, and output terminals. The storage unit 813 includes, for example, a hard disk, a RAM disk, and nonvolatile memory. The communication unit 814 includes, for example, a network interface. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0475] In the computer configured as described above, for example, the CPU 801 loads the program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executes the program, thereby performing the above-described series of processes. The RAM 803 also appropriately stores data and the like necessary for the CPU 801 to perform various processes.

[0476] For example, the program executed by the computer can be applied by being recorded in the removable medium 821 as a package medium, etc. In this case, by attaching the removable medium 821 to the drive 815 , the program can be installed in the storage unit 813 via the input / output interface 810 .

[0477] In addition, the program may also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In this case, the program may be received by the communication unit 814 and installed in the storage unit 813.

[0478] In addition, the program may be installed in the ROM 802 or the storage unit 813 in advance.

[0479] <Targets of application of this technology>

[0480] This technology can be applied to any image encoding or decoding system. That is, as long as it does not conflict with the above-described present technology, the specifications of various processes related to image encoding / decoding, such as transformation (inverse transformation), quantization (inverse quantization), encoding (decoding), and prediction, are arbitrary and are not limited to the examples described above. Furthermore, some of these processes may be omitted as long as they do not conflict with the above-described present technology.

[0481] Furthermore, the present technology can be applied to a multi-view image encoding system that encodes a multi-view image including images from multiple viewpoints (views). Furthermore, the present technology can be applied to a multi-view image decoding system that decodes encoded data of a multi-view image including images from multiple viewpoints (views). In this case, the present technology only needs to be applied to encoding and decoding for each viewpoint (view).

[0482] Furthermore, the present technology can be applied to a hierarchical image encoding (scalable encoding) system that encodes hierarchical images in layers (hierarchies) to provide a scalability function for predetermined parameters. Furthermore, the present technology can be applied to a hierarchical image decoding (scalable decoding) system that decodes encoded data of hierarchical images in layers (hierarchies) to provide a scalability function for predetermined parameters. In this case, the present technology only needs to be applied to the encoding and decoding of each layer (layer).

[0483] Furthermore, the inverse adaptive color transform device 100 , the inverse quantization and inverse transform device 200 , the image decoding device 400 , and the image encoding device 500 have been described above as application examples of the present technology, but the present technology can be applied to an arbitrary configuration.

[0484] For example, the present technology can be applied to various electronic devices, such as transmitters or receivers (e.g., television receivers or mobile phones) in satellite broadcasting, cable broadcasting such as cable television, distribution on the Internet, and distribution to terminals via cellular communications, or devices that record images on media such as optical disks, magnetic disks, and flash memories and reproduce images from storage media (e.g., hard disk recorders and camera devices).

[0485] In addition, for example, the present technology can also be implemented as a partial configuration of a device, such as a processor as a system large-scale integration (LSI) (e.g., a video processor), a module using multiple processors (e.g., a video module), a unit using multiple modules (e.g., a video unit), or a set obtained by further adding other functions to the unit (e.g., a video set).

[0486] Furthermore, for example, this technology can also be applied to a network system consisting of multiple devices. For example, this technology can be implemented as cloud computing, where multiple devices collaborate to share and process data via a network. For example, this technology can be implemented in a cloud service that provides image (moving image) related services to any terminal such as a computer, audio-visual (AV) device, portable information processing terminal, or Internet of Things (IoT) device.

[0487] Note that in this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), and it does not matter whether all components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, as well as a device in which multiple modules are housed in a single housing, are both systems.

[0488] <Fields and applications of this technology>

[0489] Systems, devices, processing units, etc. using this technology can be used in any field such as transportation, medical care, crime prevention, agriculture, animal husbandry, mining, beauty, factories, home appliances, weather and nature monitoring. In addition, its application is also arbitrary.

[0490] For example, the present technology can be applied to systems or devices that provide content for viewing, etc. Furthermore, for example, the present technology can be applied to systems and devices provided for transportation, such as traffic condition monitoring and autonomous driving control. Furthermore, for example, the present technology can be applied to systems or devices provided for safety. Furthermore, for example, the present technology can be applied to systems or devices that provide automatic control of machines, etc. Furthermore, for example, the present technology can be applied to systems and devices used for agriculture and animal husbandry. Furthermore, the present technology can be applied to systems and devices that monitor natural conditions such as volcanoes, forests and oceans, and wildlife. Furthermore, for example, the present technology can be applied to systems and devices provided for sports.

[0491] <Other>

[0492] Note that in this specification, a "flag" is information for identifying a plurality of states, and includes not only information for identifying two states of true (1) and false (0), but also information capable of identifying three or more states. Therefore, the value that the "flag" can take can be, for example, binary 1 / 0 or ternary or more. That is, the number of bits constituting the "flag" is arbitrary, and can be one bit or a plurality of bits. In addition, since it is assumed that identification information (including a flag) includes not only identification information in a bit stream, but also difference information of the identification information in the bit stream relative to specific reference information, in this specification, "flag" and "identification information" include not only information, but also difference information relative to the reference information.

[0493] In addition, various types of information (metadata, etc.) related to the coded data (bitstream) can be transmitted or recorded in any form as long as the information is associated with the coded data. Here, the term "association" means that, for example, one data can be used (linked) when processing another data. That is, mutually associated data can be collected as one data, or can be separate data. For example, information associated with the coded data (image) can be transmitted on a transmission path different from the transmission path of the coded data (image). In addition, for example, information associated with the coded data (image) can be recorded in a recording medium different from the coded data (image) (or in another recording area of ​​the same recording medium). Please note that this "association" may be a part of the data rather than the entire data. For example, an image and information corresponding to the image can be associated with each other in arbitrary units such as multiple frames, one frame, or a part of a frame.

[0494] Note that in this specification, terms such as "combine", "multiplex", "add", "integrate", "include", "store", "insert" and "insert" mean combining multiple items into one, such as combining encoded data and metadata into one data, and mean a method of the above-mentioned "association".

[0495] Furthermore, the embodiment of the present technology is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present technology.

[0496] For example, a configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, a configuration described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Furthermore, a configuration other than the above configuration may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, a portion of the configuration of a particular device (or processing unit) may be included in the configuration of another device (or another processing unit).

[0497] In addition, for example, the above-mentioned program can be executed in any device. In this case, it is sufficient that the device has necessary functions (functional blocks, etc.) and can obtain necessary information.

[0498] Furthermore, for example, each step of a flowchart may be executed by a single device, or may be shared and executed by multiple devices. Furthermore, when multiple processes are included in a single step, the multiple processes may be executed by a single device, or may be shared and executed by multiple devices. In other words, the multiple processes included in a single step may also be executed as a single process. Conversely, processes described as multiple steps may be collectively executed as a single step.

[0499] In addition, the program executed by the computer may have the following features. For example, the processing of the steps of the description program can be performed in a time series in the order described in this specification. In addition, the processing of the steps of the description program can be performed in parallel. In addition, the processing of the steps of the description program can be performed separately at necessary timing (for example, when called). That is, as long as there is no contradiction, the processing of each step can be performed in an order different from the above order. In addition, the processing of the steps of the description program can be performed in parallel with the processing of another program. In addition, the processing of the steps of the description program can be performed in combination with the processing of another program.

[0500] Furthermore, for example, multiple technologies related to the present technology may be independently implemented as a single entity, as long as no contradiction exists. Of course, multiple arbitrary technologies may be implemented in combination. For example, some or all of the technologies described in any one of the embodiments may be implemented in combination with some or all of the technologies described in other embodiments. Furthermore, some or all of any of the above technologies may be implemented in combination with other technologies not described above.

[0501] Note that the present technology can also be configured as follows.

[0502] (1) An image processing device comprising:

[0503] a cropping processing unit configured to crop coefficient data subjected to lossless adaptive color transform at a level based on a bit depth of the coefficient data; and

[0504] an inverse adaptive color transform unit configured to perform inverse adaptive color transform on the coefficient data cropped at the level by the cropping processing unit by a lossless method.

[0505] (2) The image processing device according to (1), wherein

[0506] The cropping processing unit crops the luminance component and the color component of the coefficient data at the same level as each other.

[0507] (3) The image processing device according to (2), wherein

[0508] The cropping processing unit performs the following operations:

[0509] clipping the luma and color components of the coefficient data using as upper bounds a value obtained by subtracting 1 from a power of 2 raised to the value obtained by adding 1 to the bit depth, and

[0510] The luma component and the color component of the coefficient data are clipped using a value obtained by multiplying -1 by a power of 2 with a value obtained by adding 1 to the bit depth as the power exponent as a lower bound.

[0511] (4) The image processing device according to (2), wherein

[0512] The level is a value based on the bit depth and the dynamic range of the buffer storing the coefficient data.

[0513] (5) The image processing device according to (4), wherein

[0514] The level is a value derived using a smaller value of: a value based on the bit depth but not based on the dynamic range of the buffer; and a value based on the dynamic range of the buffer instead of based on the bit depth.

[0515] (6) The image processing device according to (5), wherein

[0516] The cropping processing unit performs the following operations:

[0517] clipping the luma and color components of the coefficient data using as an upper bound a value obtained by subtracting 1 from a power of 2 raised to a power exponent of the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer, and

[0518] The luma component and the color component of the coefficient data are clipped using a value as a lower bound, the value being obtained by multiplying -1 by the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer raised to a power of 2 as a power exponent.

[0519] (7) The image processing device according to (1), wherein

[0520] The cropping processing unit performs the following operations:

[0521] clipping the luma component of the coefficient data at a first level, and

[0522] The color components of the coefficient data are clipped at a second level.

[0523] (8) The image processing device according to (7), wherein

[0524] The second level has a wider difference between the upper and lower bounds than the difference between the upper and lower bounds of the first level.

[0525] (9) The image processing device according to (8), wherein

[0526] The cropping processing unit performs the following operations:

[0527] clipping the luma component of the coefficient data using as an upper bound a value obtained by subtracting 1 from a power of 2 exponented by the bit depth,

[0528] The luma component of the coefficient data is clipped using a value obtained by multiplying -1 by a power of 2 with the bit depth as an exponent as a lower bound, wherein,

[0529] clipping the color components of the coefficient data using, as an upper bound, a value obtained by subtracting 1 from a power of 2 exponented by a value obtained by adding 1 to the bit depth, and

[0530] The color components of the coefficient data are clipped using, as a lower bound, a value obtained by multiplying -1 by a power of 2 with a value obtained by adding 1 to the bit depth as a power exponent.

[0531] (10) The image processing device according to (7), wherein

[0532] The first level and the second level are values ​​based on the bit depth and a dynamic range of a buffer storing the coefficient data.

[0533] (11) The image processing device according to (10), wherein

[0534] The cropping processing unit performs the following operations:

[0535] clipping the luma component of the coefficient data using as an upper bound a value obtained by subtracting 1 from the power of 2 as the exponent of the smaller of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer,

[0536] clipping the luma component of the coefficient data by a lower bound obtained by multiplying -1 by a power of 2 raised to the lesser of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer,

[0537] clipping the color components of the coefficient data using as an upper bound a value obtained by subtracting 1 from a power of 2 raised to the power of the smaller of the bit depth and the value obtained by subtracting 1 from the dynamic range of the buffer, and

[0538] The color components of the coefficient data are clipped using a value obtained by multiplying -1 by the smaller of a value obtained by adding 1 to the bit depth and a value obtained by subtracting 1 from the dynamic range of the buffer raised to a power of 2 as a power exponent.

[0539] (12) The image processing device according to (1), wherein

[0540] The level is a value based on the dynamic range of the buffer storing the coefficient data.

[0541] (13) The image processing device according to (12), wherein

[0542] The cropping processing unit performs the following operations:

[0543] clipping the luma and color components of the coefficient data using as an upper bound a value obtained by subtracting 1 from a power of 2 raised to a power exponent obtained by subtracting 1 from the dynamic range of the buffer, and

[0544] The luminance component and the color component of the coefficient data are clipped using a value obtained by multiplying -1 by a value obtained by subtracting 1 from the dynamic range of the buffer raised to a power of 2 as a power exponent as a lower bound.

[0545] (14) The image processing device according to (1), further comprising:

[0546] an inverse orthogonal transform unit configured to perform an inverse orthogonal transform on an orthogonal transform coefficient to generate the coefficient data subjected to the lossless adaptive color transform,

[0547] The cropping processing unit is configured to crop the coefficient data generated by the inverse orthogonal transform unit at the level.

[0548] (15) The image processing device according to (14), further comprising:

[0549] an inverse quantization unit configured to inversely quantize the quantized coefficients and generate the orthogonal transform coefficients,

[0550] The inverse orthogonal transform unit is configured to perform inverse orthogonal transform on the orthogonal transform coefficients generated by the inverse quantization unit.

[0551] (16) The image processing device according to (15), further comprising:

[0552] a decoding unit configured to decode the encoded data and generate the quantized coefficients,

[0553] The inverse quantization unit is configured to inverse quantize the quantization coefficients generated by the decoding unit.

[0554] (17) The image processing device according to (16), wherein

[0555] The inverse adaptive color transform unit is configured to perform inverse adaptive color transform on the coefficient data cropped at the level by the cropping processing unit by the lossless method to generate prediction residual data of image data, and

[0556] The image processing apparatus further includes a calculation unit configured to add prediction data of the image data to the prediction residual data generated by the inverse adaptive color transform unit to generate the image data.

[0557] (18) The image processing device according to (15), further comprising:

[0558] a transform and quantization unit configured to: perform adaptive color transform on image data by a lossless method to generate coefficient data, perform orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantize the orthogonal transform coefficients to generate quantized coefficients; and

[0559] an encoding unit configured to encode the quantized coefficients generated by the transform and quantization unit to generate encoded data,

[0560] The inverse quantization unit is configured to inverse quantize the quantization coefficients generated by the transform quantization unit.

[0561] (19) The image processing device according to (18), further comprising:

[0562] a computing unit configured to subtract prediction data of the image data from the image data to generate prediction residual data,

[0563] In which, the transform and quantization unit is configured to: perform adaptive color transform on the prediction residual data generated by the calculation unit through a lossless method to generate the coefficient data, inverse transform the coefficient data to generate the orthogonal transform coefficients, and quantize the orthogonal transform coefficients to generate the quantized coefficients.

[0564] (20) An image processing method comprising:

[0565] clipping coefficient data subjected to a lossless adaptive color transform at a level based on the bit depth of the coefficient data; and

[0566] An inverse adaptive color transform is performed on the coefficient data clipped at the level by a lossless method.

[0567] Reference Signs List

[0568] 100 Inverse Adaptive Color Transformation Device

[0569] 101 Selection Unit

[0570] 102 cropping processing units

[0571] 103 Inverse YCgCo-R conversion unit

[0572] 200 Inverse quantization and inverse transformation device

[0573] 201 Inverse Quantization Unit

[0574] 202 Inverse Orthogonal Transformation Unit

[0575] 203 Inverse Adaptive Color Transformation Unit

[0576] 400 Image Decoding Device

[0577] 401 Control Unit

[0578] 412 decoding unit

[0579] 413 Inverse quantization and inverse transform unit

[0580] 414 computing units

[0581] 415 In-loop filter unit

[0582] 416 Reorder Buffer

[0583] 417 frame memory

[0584] 418 prediction unit

[0585] 500 Image Coding Device

[0586] 501 control unit

[0587] 511 Reorder Buffer

[0588] 512 computing units

[0589] 313 Transform Quantization Unit

[0590] 514 coding units

[0591] 515 Storage Buffer

[0592] 516 Inverse quantization and inverse transform unit

[0593] 517 computing units

[0594] 518 In-loop filter unit

[0595] 519 frame memory

[0596] 520 prediction units

[0597] 521 Rate Control Unit

Claims

1. An image processing device, comprising: a decoding unit configured to decode the encoded data and generate quantized coefficients, an inverse quantization unit configured to inversely quantize the quantized coefficients generated by the decoding unit and generate orthogonal transform coefficients, an inverse orthogonal transform unit configured to perform an inverse orthogonal transform on the orthogonal transform coefficient generated by the inverse quantization unit to generate coefficient data subjected to lossless adaptive color transform, a clipping processing unit configured to clip the coefficient data using a value obtained by subtracting 1 from a power of 2 obtained by adding 1 to a bit depth of the coefficient data as an upper bound, and a value obtained by multiplying −1 by a power of 2 obtained by adding 1 to the bit depth as a lower bound; An inverse adaptive color transform unit is configured to perform inverse adaptive color transform on the coefficient data clipped by the clipping processing unit at the upper bound and the lower bound by a lossless method.

2. The image processing apparatus according to claim 1, wherein: The clipping processing unit clips the luminance component and the color component of the coefficient data with the upper bound and the lower bound being the same as each other.

3. The image processing apparatus according to claim 1, wherein: The inverse adaptive color transform unit is configured to perform inverse adaptive color transform on the coefficient data cropped by the cropping processing unit at the upper bound and the lower bound by the lossless method to generate prediction residual data of image data, and The image processing apparatus further includes a calculation unit configured to add prediction data of the image data to the prediction residual data generated by the inverse adaptive color transform unit to generate the image data.

4. An image processing method, comprising: Decoding the encoded data and generating quantized coefficients; Inverse quantizing the quantized coefficients and generating orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform, clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 whose exponent is a value obtained by adding 1 to the bit depth of the coefficient data and multiplying by −1 a value obtained by adding 1 to the power of 2 whose exponent is a value obtained by multiplying the bit depth of the coefficient data and the power of 2 as a lower bound; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using a lossless method.

5. An image processing device comprising: a transform and quantization unit configured to: perform adaptive color transform on image data by a lossless method to generate coefficient data, perform orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantize the orthogonal transform coefficients to generate quantized coefficients; an encoding unit configured to encode the quantized coefficients generated by the transform and quantization unit to generate encoded data; an inverse quantization unit configured to inversely quantize the quantization coefficient generated by the transform quantization unit and generate the orthogonal transform coefficient; an inverse orthogonal transform unit configured to perform an inverse orthogonal transform on the orthogonal transform coefficient generated by the inverse quantization unit to generate coefficient data subjected to lossless adaptive color transform, a clipping processing unit configured to clip the coefficient data using a value obtained by subtracting 1 from a power of 2 obtained by adding 1 to a bit depth of the coefficient data as an upper bound, and a value obtained by multiplying −1 by a power of 2 obtained by adding 1 to the bit depth as a lower bound; An inverse adaptive color transform unit is configured to perform inverse adaptive color transform on the coefficient data clipped by the clipping processing unit at the upper bound and the lower bound by a lossless method. The image processing apparatus according to claim 5 , wherein: The clipping processing unit clips the luminance component and the color component of the coefficient data with the upper bound and the lower bound being the same as each other.

7. The image processing apparatus according to claim 5, further comprising: a computing unit configured to subtract prediction data of the image data from the image data to generate prediction residual data, In which, the transform and quantization unit is configured to: perform adaptive color transform on the prediction residual data generated by the calculation unit through a lossless method to generate the coefficient data, perform orthogonal transform on the coefficient data to generate the orthogonal transform coefficients, and quantize the orthogonal transform coefficients to generate the quantized coefficients.

8. An image processing method, comprising: performing adaptive color transform on image data by a lossless method to generate coefficient data, performing orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantizing the orthogonal transform coefficients to generate quantized coefficients; encoding the quantized coefficients to generate encoded data; Inversely quantizing the quantized coefficients and generating the orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform; clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 as a power exponent of a value obtained by adding 1 to the bit depth of the coefficient data, and by multiplying by −1 a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth of the coefficient data, and by using as a lower bound a value obtained by multiplying −1 by a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using the lossless method.

9. A non-transitory computer-readable medium having a program embodied thereon, the program, when executed by a computer, causing the computer to perform an image processing method, the image processing method comprising: Decoding the encoded data and generating quantized coefficients; Inverse quantizing the quantized coefficients and generating orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform; clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 whose exponent is a value obtained by adding 1 to the bit depth of the coefficient data and multiplying by −1 a value obtained by adding 1 to the power of 2 whose exponent is a value obtained by multiplying the bit depth of the coefficient data and the power of 2 as a lower bound; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using a lossless method.

10. A computer program product comprising a computer program / instructions which, when executed by a computer, cause the computer to perform an image processing method, the image processing method comprising: performing adaptive color transform on image data by a lossless method to generate coefficient data, performing orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantizing the orthogonal transform coefficients to generate quantized coefficients; encoding the quantized coefficients to generate encoded data; Inversely quantizing the quantized coefficients and generating the orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform, clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 as a power exponent of a value obtained by adding 1 to the bit depth of the coefficient data, and by multiplying by −1 a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth of the coefficient data, and by using as a lower bound a value obtained by multiplying −1 by a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using the lossless method.

11. A non-transitory computer-readable medium having a program embodied thereon, the program, when executed by a computer, causing the computer to perform an image processing method, the image processing method comprising: performing adaptive color transform on image data by a lossless method to generate coefficient data, performing orthogonal transform on the coefficient data to generate orthogonal transform coefficients, and quantizing the orthogonal transform coefficients to generate quantized coefficients; encoding the quantized coefficients to generate encoded data; Inversely quantizing the quantized coefficients and generating the orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform, clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 as a power exponent of a value obtained by adding 1 to the bit depth of the coefficient data, and by multiplying by −1 a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth of the coefficient data, and by using as a lower bound a value obtained by multiplying −1 by a value obtained by raising the power of 2 as a power exponent of the value obtained by adding 1 to the bit depth; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using the lossless method.

12. A computer program product comprising a computer program / instructions which, when executed by a computer, cause the computer to perform an image processing method, the image processing method comprising: Decoding the encoded data and generating quantized coefficients; Inverse quantizing the quantized coefficients and generating orthogonal transform coefficients; performing an inverse orthogonal transform on the orthogonal transform coefficients to generate coefficient data subjected to lossless adaptive color transform; clipping the coefficient data by using as an upper bound a value obtained by subtracting 1 from a power of 2 whose exponent is a value obtained by adding 1 to the bit depth of the coefficient data and multiplying by −1 a value obtained by adding 1 to the power of 2 whose exponent is a value obtained by multiplying the bit depth of the coefficient data and the power of 2 as a lower bound; An inverse adaptive color transform is performed on the coefficient data clipped by the upper bound and the lower bound using a lossless method.

Images