Image processing device and method

Abstract

To provide an image processing device and an image processing method, suppressing increase in an encoding load.SOLUTION: An image encoding apparatus 300 which is one aspect of an image processing device includes a control unit 301 acting as an encoding mode setting unit configured to set an encoding mode of encoding of an image by setting a transform type with a minimum encoding cost in a non-joint chrominance encoding mode as a transform type in a joint chrominance encoding mode, and deriving an encoding cost in the joint chrominance encoding mode.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an image processing device and method, and more particularly to an image processing device and method capable of suppressing an increase in the encoding load. [Background technology]

[0002] Conventionally, a coding method has been proposed in which a prediction residual of a video is derived, coefficient transformed, quantized, and coded (see, for example, Non-Patent Document 1 and Non-Patent Document 2). In the Versatile Video Coding (VVC) Working Draft described in Non-Patent Document 1, chrominance transform skip can be applied regardless of the joint chrominance coding mode (Joint CbCr mode). In contrast, in the implementation of the VVC VTM software described in Non-Patent Document 2, application of chrominance transform skip is restricted in the joint chrominance coding mode.

[0003] When application of chrominance transform skip is restricted in the joint chrominance coding mode as described in Non-Patent Document 2, signaling of the transform skip flag in the joint chrominance coding mode is unnecessary. In other words, signaling of the transform skip flag in the joint chrominance coding mode may unnecessarily increase the amount of code and reduce the coding efficiency. In other words, there is a risk of reducing the coding efficiency. In contrast, in the case of the method described in Non-Patent Document 1, application of chrominance transform skip is not restricted in the joint chrominance coding mode, so that reduction in coding efficiency due to redundancy of the transform skip flag is suppressed. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Benjamin Bross, Jianle Chen, Shan Liu, Ye-Kui Wang, "Versatile Video Coding (Draft 7)", JVET-P2001-vE, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Feneva, CH, 1-11 Oct 2019 [Non-Patent Document 2] Jianle Chen, Yan Ye, Seung Hwan Kim, "Algorithm description for Versatile Video Coding and Test Model 7 (VTM 7)", JVET-P2002-v1, Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11 16th Meeting: Geneva, CH, 1-11 Oct. 2019 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the case of the method described in Non-Patent Document 1, it is necessary to evaluate the joint chrominance encoding mode both in the case where transform skip is applied and in the case where transform skip is not applied, which may increase the complexity of encoding and the load of encoding.

[0006] The present disclosure has been made in consideration of such circumstances, and makes it possible to suppress an increase in the encoding load. [Means for solving the problem]

[0007] According to one aspect of the present technology, there is provided an image processing device comprising: a setting unit that sets a transform type in a non-joint chrominance encoding mode as a transform type in a joint chrominance encoding mode; and an orthogonal transform unit that performs an orthogonal transform on coefficient data of an image in accordance with the transform type in the joint chrominance encoding mode set by the setting unit. The image processing device includes:

[0008] An image processing method according to one aspect of the present technology includes: The method includes setting a transform type in a non-joint chrominance encoding mode as a transform type in a joint chrominance encoding mode, and orthogonally transforming coefficient data of an image according to the transform type in the joint chrominance encoding mode that has been set. An image processing method.

[0009] In one aspect of the present technology, an image processing device and method include: A transform type in a non-joint chrominance encoding mode is set as a transform type in a joint chrominance encoding mode, and coefficient data of an image is orthogonally transformed in accordance with the transform type in the set joint chrominance encoding mode. [Brief description of the drawings]

[0010] [Figure 1] FIG. 13 is a diagram illustrating the setting of a conversion skip flag. [Diagram 2] FIG. 13 is a diagram illustrating an example of setting a transform skip flag for deriving a coding cost. [Diagram 3] FIG. 1 is a block diagram showing an example of the main configuration of an image encoding device. [Figure 4] 13 is a flowchart showing an example of the flow of an image encoding process. [Diagram 5] 13 is a flowchart showing an example of the flow of an encoding mode setting process. [Figure 6] 6 is a flowchart continuing from FIG. 5 , showing an example of the flow of the encoding mode setting process. [Figure 7] FIG. 2 is a block diagram showing an example of the main configuration of an image decoding device. [Figure 8] 13 is a flowchart showing an example of the flow of an image decoding process. [Figure 9] FIG. 1 is a block diagram showing an example of a main configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Modes for carrying out the present disclosure (hereinafter, referred to as embodiments) will be described below in the following order. 1. Setting the encoding mode 2. First embodiment (image encoding device) 3. Second embodiment (image decoding device) 4. Notes

[0012] <1. Encoding mode setting> <References supporting technical content and technical terminology> The scope of the disclosure of the present technology includes not only the contents described in the embodiments, but also the contents described in the following non-patent documents that were publicly known at the time of filing the application, and the contents of other documents referenced in the following non-patent documents.

[0013] Non-patent document 1: (mentioned above) Non-patent document 2: (mentioned above) Non-patent document 3: Recommendation ITU-T H.264 (04 / 2017) "Advanced video coding for generic audiovisual services", April 2017 Non-Patent Document 4: Recommendation ITU-T H.265 (02 / 18) "High efficiency video coding", February 2018

[0014] In other words, the contents described in the above non-patent documents can also be a basis for determining the support requirements. For example, even if the Quad-Tree Block Structure and QTBT (Quad Tree Plus Binary Tree) Block Structure described in the above non-patent documents are not directly described in the examples, they are considered to be within the disclosure scope of the present technology and to satisfy the support requirements of the claims. Similarly, for example, technical terms such as parsing, syntax, and semantics are considered to be within the disclosure scope of the present technology and to satisfy the support requirements of the claims, even if they are not directly described in the examples.

[0015] In addition, in this specification, a "block" (not a block indicating a processing unit) used in the description as a partial region or processing unit of an image (picture) indicates any partial region in a picture, and its size, shape, characteristics, etc. are not limited, unless otherwise specified. For example, a "block" includes any partial region (processing unit) such as a TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), CU (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), subblock, macroblock, tile, or slice described in the above-mentioned non-patent document.

[0016] In addition, when specifying the size of such a block, the block size may be specified indirectly in addition to directly. For example, the block size may be specified using identification information for identifying the size. For example, the block size may be specified by a ratio or difference with respect to the size of a reference block (for example, LCU or SCU). For example, when transmitting information specifying a block size as a syntax element or the like, the information may be used to indirectly specify the size as described above. In this way, the amount of information can be reduced, and coding efficiency may be improved. The specification of the block size also includes specification of a range of block sizes (for example, specification of a range of allowable block sizes, etc.).

[0017] In this specification, encoding includes not only the whole process of converting an image into a bit stream, but also a part of the process. For example, it includes not only a process including prediction processing, orthogonal transformation, quantization, arithmetic coding, etc., but also a process collectively referring to quantization and arithmetic coding, a process including prediction processing, quantization, and arithmetic coding, etc. Similarly, decoding includes not only the whole process of converting a bit stream into an image, but also a part of the process. For example, it includes not only a process including inverse arithmetic decoding, inverse quantization, inverse orthogonal transformation, prediction processing, etc., but also a process including inverse arithmetic decoding and inverse quantization, a process including inverse arithmetic decoding, inverse quantization, and prediction processing, etc.

[0018] <Joint chrominance encoding mode and transform skip> In VVC (Versatile Video Coding) described in Non-Patent Document 1 and Non-Patent Document 2, a transform skip flag (transform_skip_flag) is defined as flag information indicating whether or not to apply a transform skip, which is a mode for skipping (omitting) an orthogonal transform. A in Fig. 1 shows an example of the syntax of the transform skip flag for the chrominance component Cb. B in Fig. 1 shows an example of the syntax of the transform skip flag for the chrominance component Cr.

[0019] Furthermore, in the VVC described in Non-Patent Document 1 and Non-Patent Document 2, a joint chrominance coding mode (Joint CbCr mode) is provided, which is a mode in which both Cb and Cr residual samples are coded as a single transform block. In other words, the joint chrominance coding mode is a mode in which orthogonal transform coefficients from which both Cb and Cr residuals can be derived are coded. In this joint chrominance coding mode, for example, the Cb coefficient is coded. Then, during decoding, the Cr coefficient is derived using the decoded Cb coefficient. In this way, it is expected that coding efficiency can be improved.

[0020] <Increased load when setting encoding mode> However, in the implementation of the VVC VTM software described in Non-Patent Document 2, the type of transformation applied to chrominance is limited in the joint chrominance coding mode. An example is shown in FIG. 1C. tu_joint_cbcr_residual_flag is flag information indicating whether the joint chrominance coding mode is applied. When tu_joint_cbcr_residual_flag = 1, it indicates that the joint chrominance coding mode is applied. When tu_joint_cbcr_residual_flag = 0, it indicates that the joint chrominance coding mode is not applied (also called non-joint chrominance coding mode).

[0021] As shown in C of FIG. 1, in the case of a non-joint chrominance coding mode, the applicable transform types are DCT2 (Discrete Cosine Transform 2) and transform skip (TS). In contrast, in the case of a joint chrominance coding mode, the applicable transform type is only DCT2. If the application of transform skip is restricted in this way, signaling of a transform skip flag in the joint chrominance coding mode is unnecessary. In other words, signaling of a transform skip flag in the joint chrominance coding mode may unnecessarily increase the amount of code, thereby reducing coding efficiency.

[0022] In contrast, in the VVC (Versatile Video Coding) Working Draft described in Non-Patent Document 1, it is possible to apply chrominance transform skip regardless of the joint chrominance coding mode (Joint CbCr mode). An example is shown in FIG. 1D. As shown in FIG. 1D, in this case, DCT2 and transform skip are applicable regardless of the joint chrominance coding mode. Therefore, compared to the method described in Non-Patent Document 2, it is possible to suppress a decrease in coding efficiency due to redundancy of the transform skip flag.

[0023] However, in the VVC described in Non-Patent Document 1 and Non-Patent Document 2, a plurality of coding modes are prepared, and the coding mode with the smallest coding cost is selected and applied. That is, in the case of the method described in Non-Patent Document 1, it is necessary to evaluate the coding costs of both the joint chrominance coding mode and the non-joint chrominance coding mode when coding, in both cases of applying and not applying transform skip. Therefore, there is a risk that the coding complexity increases and the coding load increases.

[0024] <Use of conversion type settings> Therefore, when setting the coding mode, the transform type that results in the minimum coding cost in the non-joint chrominance coding mode is set as the transform type in the joint chrominance coding mode to obtain the coding cost in the joint chrominance coding mode. Here, the transform type may be either DCT2 or transform skip as described above. In this case, the value of the chrominance transform skip flag that results in the minimum coding cost in the non-joint chrominance coding mode may be set as the chrominance transform skip flag in the joint chrominance coding mode. An example of the syntax is shown in FIG. 2. In FIG. 2, "bestTsFlag [codedCIdx] in non-JointCbCr mode" indicates the chrominance transform skip flag that results in the minimum coding cost in the non-joint chrominance coding mode. Also, "transform_skip_flag[codedCIdx] in JointCbCr mode" indicates the chrominance transform skip flag in the joint chrominance coding mode.

[0025] For example, in an image processing method, the encoding mode for encoding an image is set by setting the transformation type that results in the minimum encoding cost in a non-joint chrominance encoding mode as the transformation type in a joint chrominance encoding mode, and deriving the encoding cost in the joint chrominance encoding mode.

[0026] For example, an image processing device may include an encoding mode setting unit that sets the encoding mode for encoding an image by setting the transformation type in a chrominance encoding mode and deriving an encoding cost in a joint chrominance encoding mode.

[0027] In this way, the transform type of the joint chrominance encoding mode can be set without searching both the DCT2 and transform skip modes in the joint chrominance encoding mode. Therefore, compared with the case where the encoding costs of both the joint chrominance encoding mode and the non-joint chrominance encoding mode are derived with and without the transform skip applied, the increase in the complexity of the encoding can be suppressed, and the increase in the encoding load can be suppressed. Therefore, for example, the transform type can be set at high speed. Also, the increase in the cost of the encoder can be suppressed.

[0028] Additionally, compared to the case where application of transform skip is restricted in the joint chrominance coding mode as described in Non-Patent Document 2, it is possible to suppress a decrease in coding efficiency.

[0029] <2. First embodiment> <Image encoding device> Fig. 3 is a block diagram showing an example of the configuration of an image encoding device, which is one aspect of an image processing device to which the present technology is applied. The image encoding device 300 shown in Fig. 3 is a device that encodes image data of a moving image. For example, the image encoding device 300 can encode image data of a moving image using an encoding method described in any of the above non-patent documents.

[0030] Note that Fig. 3 shows the main processing units (blocks) and data flows, etc., and is not limited to all that is shown in Fig. 3. In other words, in the image encoding device 300, there may be processing units that are not shown as blocks in Fig. 3, and there may be processes and data flows that are not shown as arrows or the like in Fig. 3.

[0031] As shown in FIG. 3, the image encoding device 300 has a control unit 301, a rearrangement buffer 311, a calculation unit 312, an orthogonal transformation unit 313, a quantization unit 314, an encoding unit 315, an accumulation buffer 316, an inverse quantization unit 317, an inverse orthogonal transformation unit 318, a calculation unit 319, an in-loop filter unit 320, a frame memory 321, a prediction unit 322, and a rate control unit 323.

[0032] <Control Unit> The control unit 301 divides the video data held by the rearrangement buffer 311 into blocks (CU, PU, ​​transform block, etc.) of processing units based on an external or pre-specified block size of processing units. The control unit 301 also determines coding parameters (header information Hinfo, prediction mode information Pinfo, transform information Tinfo, filter information Finfo, etc.) to be supplied to each block based on, for example, RDO (Rate-Distortion Optimization).

[0033] These coding parameters will be described in detail later. After determining the coding parameters as described above, the control unit 301 supplies them to each block. For example, the header information Hinfo is supplied to each block. The prediction mode information Pinfo is supplied to the coding unit 315 and the prediction unit 322. The transformation information Tinfo is supplied to the coding unit 315, the orthogonal transformation unit 313, the quantization unit 314, the inverse quantization unit 317, and the inverse orthogonal transformation unit 318. The filter information Finfo is supplied to the in-loop filter unit 320.

[0034] <Sorting Buffer> To the image coding device 300, each field (input image) of video image data is input in its playback order (display order). The reordering buffer 311 acquires and holds (stores) each input image in its playback order (display order). Under the control of the control unit 301, the reordering buffer 311 reorders the input images in coding order (decoding order) and divides them into blocks of processing units. The reordering buffer 311 supplies each processed input image to the calculation unit 312. The reordering buffer 311 also supplies each input image (original image) to the prediction unit 322 and the in-loop filter unit 320.

[0035] <Arithmetic section> The calculation unit 312 receives as input an image I corresponding to a block of a processing unit and a predicted image P supplied from the prediction unit 322, subtracts the predicted image P from the image I as shown in the following equation, derives a prediction residual D, and supplies it to the orthogonal transformation unit 313.

[0036] D = I - P

[0037] <Orthogonal transformation section> The orthogonal transform unit 313 executes processing related to coefficient transform. For example, the orthogonal transform unit 313 obtains the prediction residual D supplied from the calculation unit 312. In addition, the orthogonal transform unit 313 obtains the transform information Tinfo supplied from the control unit 301.

[0038] The orthogonal transform unit 313 performs an orthogonal transform on the prediction residual D based on the transform information Tinfo to derive a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient. Then, the orthogonal transform unit 313 performs a secondary transform on the primary transform coefficient to generate a secondary transform coefficient. The orthogonal transform unit 313 supplies the obtained secondary transform coefficient to the quantization unit 314 as the transform coefficient Coeff.

[0039] Note that the orthogonal transform is an example of the coefficient transform, and is not limited to this example. That is, the orthogonal transform unit 313 can perform any coefficient transform on the prediction residual D. In addition, the orthogonal transform unit 313 can perform any coefficient transform as the primary transform and the secondary transform.

[0040] <Quantization part> The quantization unit 314 executes a process related to quantization. For example, the quantization unit 314 acquires the transform coefficient Coeff supplied from the orthogonal transformation unit 313. The quantization unit 314 also acquires the transform information Tinfo supplied from the control unit 301. The quantization unit 314 then scales (quantizes) the transform coefficient Coeff based on the transform information Tinfo. Any method of quantization may be used. The rate of quantization is controlled by the rate control unit 323. The quantization unit 314 supplies the quantized transform coefficient obtained by such quantization, i.e., the quantized transform coefficient level, to the encoding unit 315 and the inverse quantization unit 317.

[0041] <Encoding part> The encoding unit 315 executes processing related to encoding. For example, the encoding unit 315 acquires a quantization transformation coefficient level level supplied from the quantization unit 314. The encoding unit 315 also acquires various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) supplied from the control unit 301. The encoding unit 315 also acquires information related to filters such as filter coefficients supplied from the in-loop filter unit 320. The encoding unit 315 also acquires information related to an optimal prediction mode supplied from the prediction unit 322.

[0042] The coding unit 315 performs variable length coding (for example, arithmetic coding) on ​​the quantized transform coefficient level LEVEL to generate a bit string (coded data). The coding unit 315 also derives residual information Rinfo from the quantized transform coefficient level LEVEL. The coding unit 315 then codes the derived residual information Rinfo to generate a bit string.

[0043] The encoding unit 315 includes, in the filter information Finfo, information on the filter supplied from the in-loop filter unit 320. The encoding unit 315 also includes, in the prediction mode information Pinfo, information on the optimal prediction mode supplied from the prediction unit 322. Then, the encoding unit 315 encodes the various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, etc.) described above to generate a bit string.

[0044] The encoding unit 315 multiplexes the bit strings of the various information generated as described above to generate encoded data. The encoding unit 315 supplies the encoded data to the accumulation buffer 316.

[0045] <Accumulation buffer> The accumulation buffer 316 temporarily holds the encoded data obtained in the encoding unit 315. The accumulation buffer 316 outputs the held encoded data, for example as a bit stream, to the outside of the image encoding device 300 at a predetermined timing. For example, this encoded data is transmitted to the decoding side via an arbitrary recording medium, an arbitrary transmission medium, an arbitrary information processing device, etc. In other words, the accumulation buffer 316 is also a transmission unit that transmits the encoded data (bit stream).

[0046] <Inverse quantization section> The inverse quantization unit 317 executes processing related to inverse quantization. For example, the inverse quantization unit 317 acquires the quantized transformation coefficient level level supplied from the quantization unit 314. In addition, the inverse quantization unit 317 acquires the transformation information Tinfo supplied from the control unit 301.

[0047] The inverse quantization unit 317 scales (inverse quantizes) the value of the quantized transform coefficient level level based on the transform information Tinfo. Note that this inverse quantization is the inverse process of the quantization performed by the quantization unit 314. The inverse quantization unit 317 supplies the transform coefficient Coeff_IQ obtained by such inverse quantization to the inverse orthogonal transform unit 318.

[0048] <Inverse orthogonal transform section> The inverse orthogonal transform unit 318 executes processing related to inverse coefficient transform. For example, the inverse orthogonal transform unit 318 obtains the transform coefficient Coeff_IQ supplied from the inverse quantization unit 317. The inverse orthogonal transform unit 318 also obtains the transform information Tinfo supplied from the control unit 301.

[0049] The inverse orthogonal transform unit 318 performs an inverse orthogonal transform on the transform coefficient Coeff_IQ based on the transform information Tinfo, and derives a prediction residual D'. Note that this inverse orthogonal transform is an inverse process of the orthogonal transform performed in the orthogonal transform unit 313. For example, the inverse orthogonal transform unit 318 performs an inverse secondary transform on the transform coefficient Coeff_IQ (secondary transform coefficient) to generate a primary transform coefficient. Also, the inverse orthogonal transform unit 318 performs an inverse primary transform on the primary transform coefficient to generate a prediction residual D'. Note that the inverse secondary transform is an inverse process of the secondary transform performed in the orthogonal transform unit 313. Also, the inverse primary transform is an inverse process of the primary transform performed in the orthogonal transform unit 313.

[0050] The inverse orthogonal transform unit 318 supplies the prediction residual D' obtained by such inverse orthogonal transform to the calculation unit 319. Note that since the inverse orthogonal transform unit 318 is similar to an inverse orthogonal transform unit (described later) on the decoding side, the explanation on the decoding side (described later) can be applied to the inverse orthogonal transform unit 318.

[0051] <Arithmetic section> The calculation unit 319 receives as input the prediction residual D' supplied from the inverse orthogonal transform unit 318 and the predicted image P supplied from the prediction unit 322. The calculation unit 319 adds the prediction residual D' to the predicted image P corresponding to the prediction residual D' to derive a local decoded image Rlocal. The calculation unit 319 supplies the derived local decoded image Rlocal to the in-loop filter unit 320 and the frame memory 321.

[0052] <In-loop filter section> The in-loop filter unit 320 executes processing related to in-loop filter processing. For example, the in-loop filter unit 320 receives as input the locally decoded image Rlocal supplied from the calculation unit 319, the filter information Finfo supplied from the control unit 301, and the input image (original image) supplied from the rearrangement buffer 311. Note that any information is input to the in-loop filter unit 320, and information other than these pieces of information may be input. For example, a prediction mode, motion information, a code amount target value, a quantization parameter QP, a picture type, information on a block (CU, CTU, etc.), etc. may be input to the in-loop filter unit 320 as necessary.

[0053] The in-loop filter unit 320 appropriately performs a filter process on the locally decoded image Rlocal based on the filter information Finfo. The in-loop filter unit 320 also uses an input image (original image) and other input information for the filter process as necessary.

[0054] For example, the in-loop filter unit 320 may apply a bilateral filter as the filtering process. Also, the in-loop filter unit 320 may apply a deblocking filter (DBF (DeBlocking Filter)) as the filtering process. Furthermore, the in-loop filter unit 320 may apply an adaptive offset filter (SAO (Sample Adaptive Offset)) as the filtering process. Also, the in-loop filter unit 320 may apply an adaptive loop filter (ALF (Adaptive Loop Filter)) as the filtering process. Furthermore, the in-loop filter unit 320 may apply a combination of a plurality of these filters as the filtering process. Note that which filter is applied and in what order are optional and can be appropriately selected. For example, the in-loop filter unit 320 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filtering process.

[0055] Of course, the filtering process performed by the in-loop filter unit 320 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 320 may apply a Wiener filter or the like.

[0056] The in-loop filter unit 320 supplies the filtered local decoded image Rlocal to the frame memory 321. When transmitting information about the filter, such as a filter coefficient, to the decoding side, the in-loop filter unit 320 supplies the information about the filter to the encoding unit 315.

[0057] <Frame memory> The frame memory 321 executes processing related to storage of image-related data. For example, the frame memory 321 receives the locally decoded image Rlocal supplied from the calculation unit 319 and the locally decoded image Rlocal that has been subjected to filtering processing supplied from the in-loop filter unit 320, and holds (stores) them. The frame memory 321 also reconstructs a decoded image R for each picture unit using the locally decoded image Rlocal, and holds it (stores it in a buffer in the frame memory 321). The frame memory 321 supplies the decoded image R (or a part thereof) to the prediction unit 322 in response to a request from the prediction unit 322.

[0058] <Prediction Department> The prediction unit 322 executes a process related to generation of a predicted image. For example, the prediction unit 322 acquires prediction mode information Pinfo supplied from the control unit 301. The prediction unit 322 also acquires an input image (original image) supplied from the sorting buffer 311. The prediction unit 322 also acquires a decoded image R (or a part thereof) to be read from the frame memory 321.

[0059] The prediction unit 322 performs prediction processing such as inter prediction or intra prediction using the prediction mode information Pinfo and an input image (original image). That is, the prediction unit 322 performs prediction and motion compensation by referring to the decoded image R as a reference image, and generates a predicted image P.

[0060] The prediction unit 322 supplies the generated predicted image P to the calculation unit 312 and the calculation unit 319. In addition, the prediction unit 322 supplies information on the prediction mode selected by the above processing, that is, the optimal prediction mode, to the encoding unit 315 as necessary.

[0061] <Rate control section> The rate control unit 323 executes a process related to rate control. For example, the rate control unit 323 controls the rate of the quantization operation of the quantization unit 314 based on the code amount of the encoded data accumulated in the accumulation buffer 316 so as to prevent overflow or underflow from occurring.

[0062] <Encoding mode control> The present technology described in <1. Setting of encoding mode> is applied to the image encoding device 300 configured as above. That is, as described above in <Reuse of transform type setting>, it is possible to apply chrominance transform skip when setting the encoding mode, regardless of the joint chrominance encoding mode. Then, the transform type that results in the smallest encoding cost in the non-joint chrominance encoding mode is set as the transform type in the joint chrominance encoding mode, and the encoding cost in the joint chrominance encoding mode is derived.

[0063] For example, the control unit 301 acts as a coding mode setting unit that sets a coding mode for coding an image. In setting the coding mode, the control unit 301 can apply chrominance transform skip regardless of the joint chrominance coding mode. Furthermore, the control unit 301 sets a transform type that results in the minimum coding cost in the non-joint chrominance coding mode as a transform type in the joint chrominance coding mode, and derives a coding cost in the joint chrominance coding mode to set the coding mode for coding an image.

[0064] In this way, the control unit 301 can set the transform type of the joint chrominance coding mode without searching both the DCT2 and transform skip modes in the joint chrominance coding mode. Therefore, the image coding device 300 can suppress an increase in the complexity of coding and an increase in the coding load, compared to a case where the application of the chrominance transform skip is not restricted in the joint chrominance coding mode as described in Non-Patent Document 1. Therefore, for example, the image coding device 300 can set the transform type at high speed. In addition, an increase in the cost of the image coding device 300 can be suppressed.

[0065] Additionally, the image coding device 300 can suppress a decrease in coding efficiency compared to a case in which application of transform skip is restricted in the joint chrominance coding mode as described in Non-Patent Document 2.

[0066] In the example shown in D of Fig. 1, a setting related to a joint chrominance encoding mode and a setting of a conversion type are executed as the setting of the encoding mode. Similarly to this example, the control unit 301 may execute a setting related to a joint chrominance encoding mode and a setting of a conversion type as the setting of the encoding mode.

[0067] For example, the control unit 301 may set whether to apply a joint chrominance encoding mode (i.e., whether to apply a joint chrominance encoding mode or a non-joint chrominance encoding mode). When applying a joint chrominance encoding mode, the control unit 301 may further set which mode among a plurality of candidates to apply (contents of joint chrominance encoding). For example, the joint chrominance encoding mode may be provided with a plurality of candidate modes, such as a mode in which the same coefficient as Cb is applied to Cr, a mode in which the sign-inverted coefficient of Cb is applied to Cr, and a mode in which the coefficient of Cb multiplied by 1 / 2 is applied to Cr. Furthermore, the control unit 301 may set a conversion type.

[0068] 1D, the transform skip may also be applied in the joint chrominance coding mode. As in this example, the control unit 301 may set whether or not to apply the transform skip as a transform type in the joint chrominance coding mode. That is, when applying the joint chrominance coding mode, the control unit 301 may set a value of a transform skip flag that is flag information indicating whether or not to apply the transform skip.

[0069] In this case, the control unit 301 may set the value of the transform skip flag (bestTsFlag) that results in the minimum coding cost in the non-joint chrominance coding mode to the transform skip flag (transform_skip_flag) in the joint chrominance coding mode, as shown in FIG.

[0070] 1D, when the joint chrominance coding mode is applied and the transform skip is not applied (non-transform skip case), the transform type is DCT2. Similarly to this example, when the control unit 301 does not apply the transform skip in the joint chrominance coding mode, the control unit 301 may apply the transform type of DCT2.

[0071] Information regarding the encoding mode set by the control unit 301 as described above is supplied as, for example, Tinfo to the orthogonal transform unit 313 and the encoding unit 315. For example, the control unit 301 supplies information such as tu_joint_residual_flag, which is flag information indicating whether or not to apply a joint chrominance encoding mode, TuCResMode, which is a parameter indicating the content of the joint chrominance encoding mode, and a transform skip flag (transform_skip_flag (also referred to as tsFlag)), as Tinfo to the orthogonal transform unit 313 and the encoding unit 315.

[0072] The orthogonal transform unit 313 performs orthogonal transform on the prediction residual D derived by the calculation unit 312 based on the information (i.e., according to the set coding mode). The quantization unit 314 quantizes the transform coefficient Coeff derived by the orthogonal transform unit 313. Furthermore, the encoding unit 315 encodes the quantized transform coefficient level level derived by the quantization unit 314 based on the information (i.e., according to the set coding mode) to generate encoded data. Furthermore, the encoding unit 315 encodes the information (e.g., a transform skip flag, etc.) and includes it in the encoded data of the quantized transform coefficient level level.

[0073] <Configuration example> These processing units (each processing unit such as the control unit 301 shown in FIG. 3) may have any configuration. For example, each processing unit may be configured with a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and by executing a program.

[0074] <Image encoding process flow> Next, an example of the flow of image encoding processing executed by the image encoding device 300 configured as above will be described with reference to the flowchart of FIG.

[0075] When the image encoding process starts, in step S301, the reordering buffer 311 is controlled by the control unit 301 to reorder the frames of the input video data from the display order to the encoding order.

[0076] In step S302, the control unit 301 sets a processing unit for the input image held by the sorting buffer 311 (divides the image into blocks).

[0077] In step S303, the control unit 301 determines (sets) coding parameters for the input image held by the reordering buffer 311.

[0078] In step S304, the prediction unit 322 performs a prediction process to generate a predicted image, etc., in an optimal prediction mode. For example, in this prediction process, the prediction unit 322 performs intra prediction to generate a predicted image, etc., in an optimal intra prediction mode. The prediction unit 322 also performs inter prediction to generate a predicted image, etc., in an optimal inter prediction mode. Furthermore, the prediction unit 322 selects an optimal prediction mode from among them based on a cost function value, etc.

[0079] In step S305, the calculation unit 312 calculates the difference between the input image and the prediction image of the optimal mode selected by the prediction process in step S304. That is, the calculation unit 312 generates a prediction residual D between the input image and the prediction image. The prediction residual D thus obtained has a reduced data amount compared to the original image data. Therefore, the data amount can be compressed compared to the case where the image is encoded as it is.

[0080] In step S306, the orthogonal transform unit 313 performs an orthogonal transform process on the prediction residual D generated by the process of step S305 to derive a transform coefficient Coeff. For example, the orthogonal transform unit 313 performs a primary transform on the prediction residual D to generate a primary transform coefficient. In addition, the orthogonal transform unit 313 performs a secondary transform on the primary transform coefficient to generate a secondary transform coefficient (transform coefficient Coeff).

[0081] In step S307, the quantization unit 314 quantizes the transform coefficient Coeff obtained by the process of step S306 by using the quantization parameter calculated by the control unit 301, etc., to derive the quantized transform coefficient level level.

[0082] In step S308, the inverse quantization unit 317 inverse quantizes the quantized transform coefficient level generated by the process of step S307 with characteristics corresponding to the quantization characteristics of step S307, to derive the transform coefficient Coeff_IQ.

[0083] In step S309, the inverse orthogonal transform unit 318 performs inverse orthogonal transform on the transform coefficient Coeff_IQ obtained by the process of step S308 using a method corresponding to the orthogonal transform process of step S306 to derive a prediction residual D'. For example, the inverse orthogonal transform unit 318 performs an inverse secondary transform on the transform coefficient Coeff_IQ (secondary transform coefficient) to generate a primary transform coefficient. The inverse orthogonal transform unit 318 also performs an inverse primary transform on the primary transform coefficient to generate a prediction residual D'.

[0084] This inverse orthogonal transform process is the same as the inverse orthogonal transform process executed on the decoding side, and therefore the description of the decoding side to be described later can be applied to the inverse orthogonal transform process of step S309.

[0085] In step S310, the calculation unit 319 generates a locally decoded image by adding the prediction image obtained by the prediction process in step S304 to the prediction residual D' derived in the process of step S309.

[0086] In step S311, the in-loop filter unit 320 performs in-loop filtering on the locally decoded image derived in the process of step S310.

[0087] In step S312, the frame memory 321 stores the locally decoded image derived by the process of step S310 and the locally decoded image filtered in step S311.

[0088] In step S313, the encoding unit 315 encodes the quantized transform coefficient level LEVEL obtained by the process of step S307. For example, the encoding unit 315 encodes the quantized transform coefficient level LEVEL, which is information related to the image, by arithmetic encoding or the like to generate encoded data. At this time, the encoding unit 315 also encodes various encoding parameters (header information Hinfo, prediction mode information Pinfo, transformation information Tinfo). Furthermore, the encoding unit 315 derives residual information RInfo from the quantized transform coefficient level LEVEL, and encodes the residual information RInfo.

[0089] In step S314, the accumulation buffer 316 accumulates the encoded data thus obtained and outputs it, for example, as a bit stream, to the outside of the image encoding device 300. This bit stream is transmitted to the decoding side, for example, via a transmission path or a recording medium. In addition, the rate control unit 323 executes rate control as necessary.

[0090] When the process of step S314 ends, the image encoding process ends.

[0091] <Encoding mode control> The present technology described in <1. Setting of encoding mode> is applied to the image encoding process with the above flow. That is, as described above in <Reuse of transform type setting>, when setting the encoding mode, it is possible to apply chrominance transform skip regardless of the joint chrominance encoding mode. Then, the transform type that results in the smallest encoding cost in the non-joint chrominance encoding mode is set as the transform type in the joint chrominance encoding mode, and the encoding cost in the joint chrominance encoding mode is derived.

[0092] For example, in step S303, the control unit 301 executes a coding mode setting process to set a coding mode for coding an image. In setting the coding mode, chrominance transform skip can be applied regardless of the joint chrominance coding mode. Furthermore, the control unit 301 sets the transform type that results in the minimum coding cost in the non-joint chrominance coding mode as the transform type in the joint chrominance coding mode, and derives the coding cost in the joint chrominance coding mode to set the coding mode.

[0093] In step S306, the orthogonal transform unit 313 performs orthogonal transform on the prediction residual D according to the set coding mode. In step S313, the coding unit 315 codes the quantized transform coefficient level level according to the set coding mode to generate coded data. Furthermore, the coding unit 315 codes information related to the coding mode (e.g., a transform skip flag, etc.) and includes it in the coded data of the quantized transform coefficient level level.

[0094] In this way, the control unit 301 can set the transform type of the joint chrominance coding mode without searching both the DCT2 and transform skip modes in the joint chrominance coding mode. Therefore, the image coding device 300 can suppress an increase in the complexity of coding and an increase in the coding load, compared to a case where the application of the chrominance transform skip is not restricted in the joint chrominance coding mode as described in Non-Patent Document 1. Therefore, for example, the image coding device 300 can set the transform type at high speed. In addition, an increase in the cost of the image coding device 300 can be suppressed.

[0095] Additionally, the image coding device 300 can suppress a decrease in coding efficiency compared to a case in which application of transform skip is restricted in the joint chrominance coding mode as described in Non-Patent Document 2.

[0096] <Encoding mode setting process flow> An example of the flow of the coding mode setting process executed in step S303 in FIG. 4 will be described with reference to the flowcharts in FIG. 5 and FIG.

[0097] When the coding mode setting process starts, the control unit 301 sets the non-joint chrominance coding mode in step S351. For example, the control unit 301 sets tu_joint_cbcr_residual_flag to false (e.g., "0"), and sets TuCResMode[xTbY][yTbY] to "0."

[0098] In step S352, the control unit 301 derives the coding cost of each transform type for the non-joint chrominance coding mode. For example, the control unit 301 derives the coding cost for the transform type DCT2 and the transform type transform skip (TS) in the non-joint chrominance coding mode. The control unit 301 executes this process for each chrominance component Cb, Cr.

[0099] In step S353, the control unit 301 sets a transform type with the smallest coding cost among the transform types derived in the process of step S352. For example, the control unit 301 sets the value of the transform skip flag corresponding to the transform type with the smallest coding cost derived in the process of step S352 to bestFlag[cIdx]. The control unit 301 executes this process for each of the color difference components Cb and Cr.

[0100] In step S354, the control unit 301 sets the joint chrominance encoding mode based on a chrominance coded block flag (cbf) in the non-joint chrominance encoding mode. The chrominance coded block flag (cbf) is flag information indicating whether or not the transform coefficient of the block is to be coded. In other words, the chrominance coded block flag (cbf) is flag information indicating whether or not the block includes a transform coefficient that is not zero.

[0101] For example, the control unit 301 sets tu_joint_cbcr_residual_flag to true (e.g., "1"). Then, the control unit 301 sets TuCResMode[xTbY][yTbY] based on tu_cbf_cb, which is the cbf of the TU to be processed of the chrominance component Cb, and tu_cbf_cr, which is the cbf of the TU to be processed of the chrominance component Cr.

[0102] For example, if tu_cbf_cb == 1 and tu_cbf_cr == 0, the control unit 301 sets TuCResMode[xTbY][yTbY] to "1." Furthermore, if tu_cbf_cb == 1 and tu_cbf_cr == 1, the control unit 301 sets TuCResMode[xTbY][yTbY] to "2." Furthermore, if tu_cbf_cb == 0 and tu_cbf_cr == 1, the control unit 301 sets TuCResMode[xTbY][yTbY] to "3."

[0103] When the process of step S354 ends, the process proceeds to Fig. 6. In step S361 of Fig. 6, the control unit 301 sets the coding component identifier codedCIdx based on the joint chrominance coding mode set in step S354. For example, if TuCResMode[xTbY][yTbY] is "1" or "2", the control unit 301 sets codedCIdx to "1" (i.e., Cb), and otherwise sets codedCIdx to "2" (i.e., Cr).

[0104] In step S362, the control unit 301 sets bestTsFlag[cIdx] set in step S353 to the transform skip flag tsFlag[codedCIdx] in the joint chrominance encoding mode (tsFlag[codedCIdx]=bestTsFlag[cIdx]).

[0105] In step S363, the control unit 301 derives the coding cost of the joint chrominance coding mode. As described above, in step S362, the transform skip flag in the joint chrominance coding mode is set to the value of the transform skip flag corresponding to the transform type that has the smallest coding cost in the non-joint chrominance coding mode. Therefore, for the joint chrominance coding mode, the control unit 301 only needs to derive the coding cost of the mode corresponding to the value of the transform skip flag. That is, in this case, the control unit 301 does not need to derive the coding cost for both the case where the transform skip is applied and the case where it is not applied. Therefore, the control unit 301 can more easily derive the coding cost for the joint chrominance coding mode.

[0106] In step S364, the control unit 301 compares the minimum coding cost of the non-joint chrominance coding modes with the coding cost of the joint chrominance coding modes, and selects the mode with the minimum coding cost.

[0107] When the process of step S364 ends, the encoding mode setting process ends, and the process returns to FIG.

[0108] In this way, the image encoding device 300 can suppress an increase in the complexity of encoding and an increase in the encoding load, compared to a case in which the application of chrominance transform skip is not restricted in the joint chrominance encoding mode as described in Non-Patent Document 1. Therefore, for example, the image encoding device 300 can set the transform type at high speed. Also, an increase in the cost of the image encoding device 300 can be suppressed.

[0109] Additionally, the image coding device 300 can suppress a decrease in coding efficiency compared to a case in which application of transform skip is restricted in the joint chrominance coding mode as described in Non-Patent Document 2.

[0110] <3. Second embodiment> <Image Decoding Device> Fig. 7 is a block diagram showing an example of a configuration of an image decoding device, which is one aspect of an image processing device to which the present technology is applied. The image decoding device 400 shown in Fig. 7 is a device that encodes encoded data of a moving image. For example, the image decoding device 400 can decode encoded data by a decoding method described in any of the above-mentioned non-patent documents. For example, the image decoding device 400 decodes encoded data (bit stream) generated by the above-mentioned image encoding device 300.

[0111] Note that Fig. 7 shows the main processing units (blocks) and data flows, etc., and is not limited to all that is shown in Fig. 7. That is, in the image decoding device 400, there may be processing units that are not shown as blocks in Fig. 7, and there may be processes and data flows that are not shown as arrows or the like in Fig. 7.

[0112] 7, the image decoding device 400 includes an accumulation buffer 411, a decoding unit 412, an inverse quantization unit 413, an inverse orthogonal transformation unit 414, a calculation unit 415, an in-loop filter unit 416, a rearrangement buffer 417, a frame memory 418, and a prediction unit 419. The prediction unit 419 includes an intra prediction unit and an inter prediction unit, not shown. The image decoding device 400 is a device for generating moving image data by decoding encoded data (bit stream).

[0113] <Accumulation buffer> The accumulation buffer 411 acquires and holds (stores) the bitstream input to the image decoding device 400. The accumulation buffer 411 supplies the accumulated bitstream to the decoding unit 412 at a predetermined timing or when a predetermined condition is met.

[0114] <Decryption section> The decoding unit 412 executes a process for decoding an image. For example, the decoding unit 412 obtains a bit stream supplied from the accumulation buffer 411. The decoding unit 412 also performs variable-length decoding of the syntax values ​​of each syntax element from the bit string in accordance with the definition of the syntax table, and derives parameters.

[0115] The parameters derived from the syntax elements and the syntax values ​​of the syntax elements include, for example, header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, residual information Rinfo, filter information Finfo, and other information. That is, the decoding unit 412 parses (analyzes and obtains) these pieces of information from the bitstream. These pieces of information will be described below.

[0116] <Header information Hinfo> The header information Hinfo includes header information such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a slice header (SH), etc. The header information Hinfo includes information that specifies, for example, an image size (horizontal width PicWidth, vertical width PicHeight), bit depth (luminance bitDepthY, chrominance bitDepthC), a chrominance array type ChromaArrayType, a maximum value MaxCUSize and a minimum value MinCUSize of a CU size, a maximum depth MaxQTDepth and a minimum depth MinQTDepth of a quad-tree partitioning (also called quad-tree partitioning), a maximum depth MaxBTDepth and a minimum depth MinBTDepth of a binary-tree partitioning (binary-tree partitioning), a maximum value MaxTSSize of a transform skip block (also called maximum transform skip block size), an on / off flag (also called a valid flag) of each encoding tool, etc.

[0117] For example, the on / off flag of the encoding tool included in the header information Hinfo includes on / off flags related to the conversion process and quantization process shown below. The on / off flag of the encoding tool can also be interpreted as a flag indicating whether or not syntax related to the encoding tool is present in the encoded data. When the value of the on / off flag is 1 (true), it indicates that the encoding tool is usable. When the value of the on / off flag is 0 (false), it indicates that the encoding tool is not usable. The interpretation of the flag value may be reversed.

[0118] For example, the header information Hinfo may include an inter-component prediction enable flag (ccp_enabled_flag). The inter-component prediction enable flag is flag information indicating whether inter-component prediction (CCP (Cross-Component Prediction), also referred to as CC prediction) is available or not. For example, when this flag information is "1" (true), it indicates that it is available, and when it is "0" (false), it indicates that it is not available.

[0119] This CCP is also called component-to-component linear prediction (CCLM or CCLMP).

[0120] <Prediction mode information Pinfo> The prediction mode information Pinfo includes, for example, size information PBSize (prediction block size) of the processing target PB (prediction block), intra prediction mode information IPinfo, motion prediction information MVinfo, and other information.

[0121] The intra prediction mode information IPinfo may include, for example, prev_intra_luma_pred_flag, mpm_idx, and rem_intra_pred_mode in JCTVC-W1005, 7.3.8.5 Coding Unit syntax, and a luma intra prediction mode IntraPredModeY derived from that syntax.

[0122] Furthermore, the intra prediction mode information IPinfo may include, for example, an inter-component prediction flag (ccp_flag (cclmp_flag)). The inter-component prediction flag (ccp_flag (cclmp_flag)) is flag information indicating whether inter-component linear prediction is applied or not. For example, when ccp_flag==1, it indicates that inter-component prediction is applied, and when ccp_flag==0, it indicates that inter-component prediction is not applied.

[0123] Furthermore, the intra-prediction mode information IPinfo may include a multi-class linear prediction mode flag (mclm_flag). The multi-class linear prediction mode flag (mclm_flag) is information on the mode of linear prediction (linear prediction mode information). More specifically, the multi-class linear prediction mode flag (mclm_flag) is flag information indicating whether or not to use the multi-class linear prediction mode. For example, "0" indicates a one-class mode (single class mode) (e.g., CCLMP), and "1" indicates a two-class mode (multi-class mode) (e.g., MCLMP).

[0124] Furthermore, the intra prediction mode information IPinfo may include a chroma sample location type identifier (chroma_sample_loc_type_idx). The chroma sample location type identifier (chroma_sample_loc_type_idx) is an identifier for identifying the type of pixel location of a chroma component (also referred to as a chroma sample location type). For example, when the chroma array type (ChromaArrayType), which is information related to the color format, indicates the 420 format, the chroma sample location type identifier is assigned as shown in the following formula.

[0125] chroma_sample_loc_type_idx == 0:Type2 chroma_sample_loc_type_idx == 1:Type3 chroma_sample_loc_type_idx == 2:Type0 chroma_sample_loc_type_idx == 3:Type1

[0126] It should be noted that this chrominance sample location type identifier (chroma_sample_loc_type_idx) is transmitted as information on the pixel location of the chrominance component (chroma_sample_loc_info()) (i.e., stored in information on the pixel location of the chrominance component).

[0127] Furthermore, the intra prediction mode information IPinfo may include a chroma MPM identifier (chroma_mpm_idx). The chroma MPM identifier (chroma_mpm_idx) is an identifier indicating which prediction mode candidate in the chroma intra prediction mode candidate list (intraPredModeCandListC) is to be specified as a chroma intra prediction mode.

[0128] Furthermore, the intra-prediction mode information IPinfo may include a luma intra-prediction mode (IntraPredModeC) derived from these syntaxes.

[0129] The motion prediction information MVinfo includes information such as merge_idx, merge_flag, inter_pred_idc, ref_idx_LX, mvp_lX_flag, X={0,1}, mvd, etc. (see, for example, JCTVC-W1005, 7.3.8.6 Prediction Unit Syntax).

[0130] Of course, the prediction mode information Pinfo may include any information, and may include information other than the above information.

[0131] <Conversion information Tinfo> The conversion information Tinfo may include, for example, the width size TBWSize and the height TBHSize of the conversion block to be processed. Note that, instead of the width size TBWSize of the conversion block to be processed, a logarithmic value log2TBWSize with a base of 2 may be applied. Also, instead of the height TBHSize of the conversion block to be processed, a logarithmic value log2TBHSize with a base of 2 may be applied.

[0132] The transform information Tinfo may also include a transform skip flag (transform_skip_flag (or ts_flag)). This transform skip flag is a flag indicating whether or not to skip the coefficient transform (or the inverse coefficient transform). Note that this transform skip flag may be signaled for each of the Y, Cb, and Cr components (transform_skip_flag[0], transform_skip_flag[1], transform_skip_flag[2]).

[0133] Furthermore, the conversion information Tinfo may include parameters such as a scan identifier (scanIdx), a quantization parameter (qp), and a quantization matrix (scaling_matrix (eg, JCTVC-W1005, 7.3.4 Scaling list data syntax)).

[0134] Of course, the information included in the conversion information Tinfo is arbitrary, and information other than the above information may be included.

[0135] <Residual information Rinfo> The residual information Rinfo (see, for example, 7.3.8.11 Residual Coding syntax of JCTVC-W1005) may include, for example, a residual data presence / absence flag (cbf (coded_block_flag)). The residual information Rinfo may also include a last non-zero coefficient X coordinate (last_sig_coeff_x_pos) and a last non-zero coefficient Y coordinate (last_sig_coeff_y_pos). The residual information Rinfo may also include a sub-block non-zero coefficient presence / absence flag (coded_sub_block_flag) and a non-zero coefficient presence / absence flag (sig_coeff_flag).

[0136] The residual information Rinfo may also include a GR1 flag (gr1_flag) which is a flag indicating whether the level of the non-zero coefficient is greater than 1, and a GR2 flag (gr2_flag) which is a flag indicating whether the level of the non-zero coefficient is greater than 2. The residual information Rinfo may also include a sign code (sign_flag) which is a code indicating the positive or negative of the non-zero coefficient. The residual information Rinfo may also include a non-zero coefficient residual level (coeff_abs_level_remaining) which is the residual level of the non-zero coefficient.

[0137] Of course, the information included in the residual information Rinfo is arbitrary, and information other than the above information may be included.

[0138] <Filter information Finfo> The filter information Finfo includes control information related to filter processing. For example, the filter information Finfo may include control information related to a deblocking filter (DBF). The filter information Finfo may also include control information related to a pixel adaptive offset (SAO). The filter information Finfo may also include control information related to an adaptive loop filter (ALF). The filter information Finfo may also include control information related to other linear filters and nonlinear filters.

[0139] For example, the filter information Finfo may include information specifying a picture to which each filter is applied or an area within the picture. The filter information Finfo may also include filter On control information or Off control information for each CU. The filter information Finfo may also include filter On control information or Off control information related to the boundary between slices or tiles.

[0140] Of course, the information included in the filter information Finfo is arbitrary, and information other than the above information may be included.

[0141] Returning to the description of the decoding unit 412, the decoding unit 412 derives the quantized transform coefficient level "level" of each coefficient position in each transform block by referring to the residual information Rinfo. The decoding unit 412 supplies the quantized transform coefficient level "level" to the inverse quantization unit 413.

[0142] Furthermore, the decoding unit 412 supplies the parsed header information Hinfo, prediction mode information Pinfo, quantization transformation coefficient level level, transformation information Tinfo, and filter information Finfo to each block.

[0143] The header information Hinfo is supplied to the inverse quantization unit 413, the inverse orthogonal transformation unit 414, the prediction unit 419, and the in-loop filter unit 416. The prediction mode information Pinfo is supplied to the inverse quantization unit 413 and the prediction unit 419. The transformation information Tinfo is supplied to the inverse quantization unit 413 and the inverse orthogonal transformation unit 414. The filter information Finfo is supplied to the in-loop filter unit 416.

[0144] Of course, the above example is merely an example, and the present invention is not limited to this example. For example, each encoding parameter may be supplied to an arbitrary processing unit. In addition, other information may be supplied to an arbitrary processing unit.

[0145] <Inverse quantization section> The inverse quantization unit 413 executes processing related to inverse quantization. For example, the inverse quantization unit 413 acquires the transformation information Tinfo and the quantized transformation coefficient level level supplied from the decoding unit 412. Furthermore, the inverse quantization unit 413 scales (inverse quantizes) the value of the quantized transformation coefficient level level based on the transformation information Tinfo, and derives the transformation coefficient Coeff_IQ after inverse quantization.

[0146] Note that this inverse quantization is performed as an inverse process of the quantization performed by the quantization unit 314 of the image encoding device 300. Also, this inverse quantization is the same process as the inverse quantization performed by the inverse quantization unit 317 of the image encoding device 300. In other words, the inverse quantization unit 317 performs the same process (inverse quantization) as the inverse quantization unit 413.

[0147] The inverse quantization unit 413 supplies the derived transformation coefficient Coeff_IQ to the inverse orthogonal transformation unit 414 .

[0148] <Inverse orthogonal transform section> The inverse orthogonal transform unit 414 executes processing related to inverse orthogonal transform. For example, the inverse orthogonal transform unit 414 obtains the transform coefficient Coeff_IQ supplied from the inverse quantization unit 413. The inverse orthogonal transform unit 414 also obtains the transform information Tinfo supplied from the decoding unit 412.

[0149] The inverse orthogonal transform unit 414 performs an inverse orthogonal transform process on the transform coefficient Coeff_IQ based on the transform information Tinfo to derive a prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficient Coeff_IQ to generate a primary transform coefficient. The inverse orthogonal transform unit 414 also performs an inverse primary transform on the primary transform coefficient to generate a prediction residual D'.

[0150] This inverse orthogonal transform is executed as an inverse process of the orthogonal transform performed by the orthogonal transform unit 313 of the image encoding device 300. This inverse orthogonal transform is also the same process as the inverse orthogonal transform performed by the inverse orthogonal transform unit 318 of the image encoding device 300. In other words, the inverse orthogonal transform unit 318 executes the same process (inverse orthogonal transform) as the inverse orthogonal transform unit 414.

[0151] The inverse orthogonal transform unit 414 supplies the derived prediction residual D′ to the calculation unit 415 .

[0152] <Arithmetic section> The calculation unit 415 executes processing related to addition of information related to an image. For example, the calculation unit 415 obtains a prediction residual D' supplied from the inverse orthogonal transform unit 414. The calculation unit 415 also obtains a predicted image P supplied from the prediction unit 419. As shown in the following formula, the calculation unit 415 adds the prediction residual D' and a predicted image P (prediction signal) corresponding to the prediction residual D' to derive a locally decoded image Rlocal.

[0153] Rlocal = D' + P

[0154] The calculation unit 415 supplies the derived locally decoded image Rlocal to the in-loop filter unit 416 and the frame memory 418.

[0155] <In-loop filter section> The in-loop filter unit 416 executes processing related to in-loop filter processing. For example, the in-loop filter unit 416 acquires a local decoded image Rlocal supplied from the calculation unit 415. The in-loop filter unit 416 also acquires filter information Finfo supplied from the decoding unit 412. Note that any information is input to the in-loop filter unit 416, and information other than the above information may be input.

[0156] The in-loop filter unit 416 appropriately performs a filter process on the local decoded image Rlocal based on the filter information Finfo. For example, the in-loop filter unit 416 may apply a bilateral filter as the filter process. Also, the in-loop filter unit 416 may apply a deblocking filter as the filter process. Furthermore, the in-loop filter unit 416 may apply an adaptive offset filter as the filter process. Also, the in-loop filter unit 416 may apply an adaptive loop filter as the filter process. Furthermore, the in-loop filter unit 416 may apply a combination of a plurality of these filters as the filter process. Note that which filter is applied and in what order are optional and can be appropriately selected. For example, the in-loop filter unit 416 applies four in-loop filters, a bilateral filter, a deblocking filter, an adaptive offset filter, and an adaptive loop filter, in this order as the filter process.

[0157] The in-loop filter unit 416 performs a filter process corresponding to the filter process performed by the encoding side (for example, the in-loop filter unit 320 of the image encoding device 300). Of course, the filter process performed by the in-loop filter unit 416 is arbitrary and is not limited to the above example. For example, the in-loop filter unit 416 may apply a Wiener filter or the like.

[0158] The in-loop filter unit 416 supplies the filtered locally decoded image Rlocal to a reordering buffer 417 and a frame memory 418.

[0159] <Sorting Buffer> The reordering buffer 417 receives the locally decoded image Rlocal supplied from the in-loop filter unit 416 and holds (stores) it. The reordering buffer 417 reconstructs a decoded image R for each picture using the locally decoded image Rlocal and holds it (stores it in the buffer). The reordering buffer 417 reorders the obtained decoded images R from the decoding order to the playback order. The reordering buffer 417 outputs the reordered decoded images R to the outside of the image decoding device 400 as video data.

[0160] <Frame memory> The frame memory 418 executes processing related to storage of image-related data. For example, the frame memory 418 acquires a locally decoded image Rlocal supplied from the calculation unit 415. Then, the frame memory 418 reconstructs a decoded image R for each picture unit using the locally decoded image Rlocal. The frame memory 418 stores the reconstructed decoded image R in a buffer within the frame memory 418.

[0161] The frame memory 418 also acquires the in-loop filtered locally decoded image Rlocal supplied from the in-loop filter unit 416. The frame memory 418 then reconstructs a decoded image R for each picture using the in-loop filtered locally decoded image Rlocal. The frame memory 418 stores the reconstructed decoded image R in a buffer within the frame memory 418.

[0162] Furthermore, the frame memory 418 appropriately supplies the stored decoded image R (or a part thereof) to the prediction unit 419 as a reference image.

[0163] The frame memory 418 may store header information Hinfo, prediction mode information Pinfo, transformation information Tinfo, filter information Finfo, and the like, which are related to the generation of a decoded image.

[0164] <Prediction Department> The prediction unit 419 executes a process related to the generation of a predicted image. For example, the prediction unit 419 acquires prediction mode information Pinfo supplied from the decoding unit 412. The prediction unit 419 also executes a prediction process by a prediction method specified by the prediction mode information Pinfo, and derives a predicted image P. When deriving the predicted image P, the prediction unit 419 uses a decoded image R (or a part thereof) stored in the frame memory 418, which is specified by the prediction mode information Pinfo, as a reference image. This decoded image R may be an unfiltered image or may be a filtered image. The prediction unit 419 supplies the derived predicted image P to the calculation unit 415.

[0165] <Configuration example> In addition, these processing units (accumulation buffer 411 to prediction unit 419) have any configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Also, each processing unit may have, for example, a CPU, a ROM, a RAM, etc., and may realize the above-mentioned processing by executing a program using them. Of course, each processing unit may have both configurations, and may realize part of the above-mentioned processing by a logic circuit and the other by executing a program. The configurations of each processing unit may be independent of each other. For example, some processing units may realize part of the above-mentioned processing by a logic circuit, other processing units may realize the above-mentioned processing by executing a program, and still other processing units may realize the above-mentioned processing by both a logic circuit and executing a program.

[0166] <Flow of image decoding process> Next, an example of the flow of image decoding processing executed by the image decoding device 400 configured as above will be described with reference to the flowchart in FIG.

[0167] When the image decoding process starts, in step S401, the accumulation buffer 411 acquires and holds (accumulates) encoded data (bit stream) supplied from outside the image decoding device 400.

[0168] In step S402, the decoding unit 412 decodes the coded data (bit stream) to obtain the quantized transform coefficient level level. Furthermore, the decoding unit 412 parses (analyzes and obtains) various coding parameters from the coded data (bit stream) through this decoding.

[0169] In step S403, the inverse quantization unit 413 performs inverse quantization, which is the inverse process of the quantization performed on the encoding side, on the quantized transform coefficient level obtained by the process in step S402 to obtain a transform coefficient Coeff_IQ.

[0170] In step S404, the inverse orthogonal transform unit 414 performs an inverse orthogonal transform process, which is an inverse process of the orthogonal transform process performed on the encoding side, on the transform coefficient Coeff_IQ obtained in step S403, to obtain a prediction residual D'. For example, the inverse orthogonal transform unit 414 performs an inverse secondary transform on the transform coefficient Coeff_IQ (secondary transform coefficient) to generate a primary transform coefficient. The inverse orthogonal transform unit 414 also performs an inverse primary transform on the primary transform coefficient to generate a prediction residual D'.

[0171] In step S405, the prediction unit 419 performs a prediction process using a prediction method specified by the encoding side based on the information parsed in step S402, and generates a predicted image P by, for example, referring to a reference image stored in the frame memory 418.

[0172] In step S406, the calculation unit 415 adds the prediction residual D' obtained in step S404 and the predicted image P obtained in step S405 to derive a locally decoded image Rlocal.

[0173] In step S407, the in-loop filter unit 416 performs in-loop filtering on the locally decoded image Rlocal obtained by the processing in step S406.

[0174] In step S408, the reordering buffer 417 derives the decoded image R using the “filtered locally decoded image Rlocal” obtained by the process in step S407, and reorders the order of the decoded images R from the decoding order to the playback order. The decoded images R reordered in the playback order are output to the outside of the image decoding device 400 as a moving image.

[0175] Furthermore, in step S409, the frame memory 418 stores at least one of the locally decoded image Rlocal obtained by the process in step S406 and the locally decoded image Rlocal after the filter process obtained by the process in step S407.

[0176] When the process of step S409 ends, the image decoding process ends.

[0177] <4. Notes> <Computer> The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the programs constituting the software are installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, etc., capable of executing various functions by installing various programs.

[0178] FIG. 9 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes by a program.

[0179] In a computer 800 shown in FIG. 9, a CPU (Central Processing Unit) 801, a ROM (Read Only Memory) 802, and a RAM (Random Access Memory) 803 are interconnected via a bus 804.

[0180] An input / output interface 810 is also connected to the bus 804. To the input / output interface 810, an input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected.

[0181] The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, etc. The output unit 812 includes, for example, a display, a speaker, an output terminal, etc. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, etc. The communication unit 814 includes, for example, a network interface. The drive 815 drives removable media 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0182] In the computer configured as above, the series of processes described above are executed by the CPU 801, for example, loading a program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executing the program. The RAM 803 also stores data and the like necessary for the CPU 801 to execute various processes as appropriate.

[0183] The program executed by the computer can be applied by recording it on a removable medium 821 such as a package medium, for example. In this case, the program can be installed in the storage unit 813 via the input / output interface 810 by mounting the removable medium 821 on the drive 815.

[0184] This program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, digital satellite broadcasting, etc. In this case, the program can be received by the communication unit 814 and installed in the storage unit 813.

[0185] In addition, this program can also be installed in advance in the ROM 802 or the storage unit 813.

[0186] <Target of application of this technology> This technology can be applied to any image coding method. In other words, as long as it does not contradict the above-mentioned technology, the specifications of various processes related to image coding, such as transform (inverse transform), quantization (inverse quantization), coding, prediction, etc., are arbitrary and are not limited to the above-mentioned examples. In addition, as long as it does not contradict the above-mentioned technology, some of these processes may be omitted.

[0187] In addition, the present technology can be applied to a multi-view image encoding system (or a multi-view image decoding system) that performs encoding and decoding of a multi-view image including images of a plurality of views. In that case, the present technology may be applied in encoding and decoding of each view.

[0188] Furthermore, the present technology can be applied to a hierarchical image coding (scalable coding) system (or a hierarchical image decoding system) that performs coding of a hierarchical image that is layered so as to have a scalability function for a predetermined parameter. In this case, the present technology may be applied in the coding or decoding of each layer.

[0189] Furthermore, although the image encoding device 300 and the image decoding device 400 have been described above as application examples of the present technology, the present technology can be applied to any configuration.

[0190] For example, the present technology can be applied to various electronic devices, such as transmitters and receivers (e.g., television sets and mobile phones) in satellite broadcasting, wired broadcasting such as cable TV, distribution over the Internet, and distribution to terminals via cellular communication, or devices (e.g., hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and play images from these storage media.

[0191] Furthermore, for example, the present technology can also be implemented as part of an apparatus, such as a processor (e.g., a video processor) as a system LSI (Large Scale Integration), a module (e.g., a video module) using multiple processors, a unit (e.g., a video unit) using multiple modules, or a set in which other functions are added to a unit (e.g., a video set).

[0192] In addition, for example, the present technology can also be applied to a network system composed of multiple devices. For example, the present technology may be implemented as cloud computing in which multiple devices share and cooperate with each other to perform processing via a network. For example, the present technology may be implemented in a cloud service that provides services related to images (video images) to any terminal such as a computer, an AV (Audio Visual) device, a portable information processing terminal, or an IoT (Internet of Things) device.

[0193] In this specification, a system refers to a collection of multiple components (devices, modules (parts), etc.), regardless of whether all the components are in the same case. Therefore, multiple devices housed in separate cases and connected via a network, and a single device in which multiple modules are housed in a single case, are both systems.

[0194] <Fields and uses where this technology can be applied> Systems, devices, processing units, etc. to which the present technology is applied can be used in any field, such as transportation, medical care, crime prevention, agriculture, livestock farming, mining, beauty, factories, home appliances, weather, and nature monitoring. In addition, the applications are also arbitrary.

[0195] For example, the present technology can be applied to systems and devices used for providing ornamental content, etc. Furthermore, for example, the present technology can be applied to systems and devices used for traffic, such as monitoring traffic conditions and automatic driving control. Furthermore, for example, the present technology can be applied to systems and devices used for security. Furthermore, for example, the present technology can be applied to systems and devices used for automatic control of machines, etc. Furthermore, for example, the present technology can be applied to systems and devices used for agriculture and livestock farming. Furthermore, for example, the present technology can be applied to systems and devices that monitor natural conditions, such as volcanoes, forests, and oceans, and wildlife, etc. Furthermore, for example, the present technology can be applied to systems and devices used for sports.

[0196] <Other> In this specification, a "flag" refers to information for identifying multiple states, and includes not only information used to identify two states, true (1) or false (0), but also information capable of identifying three or more states. Thus, the value that this "flag" can take may be, for example, a binary value of 1 / 0, or may be three or more values. In other words, the number of bits constituting this "flag" is arbitrary, and may be one bit or multiple bits. Furthermore, it is assumed that identification information (including flags) is not only included in the bit stream, but also includes difference information of the identification information with respect to certain reference information in the bit stream. Therefore, in this specification, "flag" and "identification information" include not only the information itself, but also difference information with respect to the reference information.

[0197] Also, various information (metadata, etc.) related to the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, that one piece of data can be used (linked) when processing the other piece of data. That is, data associated with each other may be combined into one piece of data, or each piece of data may be made into separate data. For example, information associated with coded data (image) may be transmitted on a transmission path different from that of the coded data (image). Also, for example, information associated with coded data (image) may be recorded on a recording medium different from that of the coded data (image) (or on a different recording area of ​​the same recording medium). Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in any unit, such as multiple frames, one frame, or a part of a frame.

[0198] In this specification, the terms "combine," "multiplex," "append," "integrate," "include," "store," "embed," "insert," and the like refer to combining multiple items into one, such as combining encoded data and metadata into one piece of data, and refer to one method of "associating" as described above.

[0199] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0200] For example, the configuration described above as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, the configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or processing unit). Furthermore, as long as the configuration and operation of the system as a whole are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0201] Also, for example, the above-mentioned program may be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0202] Also, for example, each step of one flowchart may be executed by one device, or may be shared and executed by multiple devices. Furthermore, when one step includes multiple processes, the multiple processes may be executed by one device, or may be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as one step.

[0203] A program executed by a computer may, for example, be configured so that the processing of the steps describing the program is executed in chronological order according to the order described in this specification. Also, a program executed by a computer may be configured so that the processing of the steps describing the program is executed in parallel. Furthermore, a program executed by a computer may be configured so that the processing of the steps describing the program is executed individually at a necessary timing, such as when the program is called. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps describing the program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0204] Also, for example, a plurality of techniques related to the present technology can be implemented independently and independently, as long as no contradiction occurs. Of course, any of a plurality of the present technologies can be implemented in combination. For example, a part or all of the present technology described in any of the embodiments can be implemented in combination with a part or all of the present technology described in other embodiments. Also, a part or all of any of the above-mentioned present technologies can be implemented in combination with other technologies not described above.

[0205] The present technology can also be configured as follows. (1) a coding mode setting unit that sets a transformation type resulting in a minimum coding cost in a non-joint chrominance coding mode as a transformation type in a joint chrominance coding mode, and derives a coding cost in the joint chrominance coding mode, thereby setting a coding mode for coding an image. An image processing device comprising: (2) The encoding mode setting unit sets the joint color difference encoding mode and the conversion type as the encoding mode setting. An image processing device as described in (1). (3) The encoding mode setting unit sets whether to apply a transform skip as the transform type in the joint chrominance encoding mode. An image processing device as described in (2). (4) When the transform skip is not applied, the encoding mode setting unit applies DCT2 as the transform type. An image processing device as described in (3). (5) The coding mode setting unit sets a value of a transform skip flag that results in a minimum coding cost in the non-joint chrominance coding mode to the transform skip flag in the joint chrominance coding mode. An image processing device according to (3) or (4). (6) The encoding mode setting unit sets the joint chrominance encoding mode based on a chrominance encoding block flag in the non-joint chrominance encoding mode. An image processing device according to any one of (2) to (5). (7) The encoding mode setting unit sets an encoding component identifier based on the set joint chrominance encoding mode. An image processing device according to (6). (8) The coding mode setting unit derives a coding cost of each transform type in the non-joint chrominance coding mode, sets the transform type having the smallest coding cost among the derived transform types, and sets the set transform type as the transform type in the joint chrominance coding mode. An image processing device according to any one of (1) to (7). (9) The coding mode setting unit compares a minimum coding cost of the non-joint chrominance coding mode with a coding cost of the joint chrominance coding mode, and selects a mode with a minimum coding cost. An image processing device according to any one of (1) to (8). (10) The image processing apparatus further includes an orthogonal transform unit that performs an orthogonal transform on coefficient data of the image in accordance with the coding mode set by the coding mode setting unit. An image processing device according to any one of (1) to (9). (11) The image forming apparatus further includes a coding unit that codes the coefficient data orthogonally transformed by the orthogonal transform unit in accordance with the coding mode set by the coding mode setting unit. The image processing device according to (10). (12) The coding mode setting unit sets a transform skip flag indicating whether to apply a transform skip as the coding mode, The encoding unit encodes the transform skip flag set by the encoding mode setting unit. The image processing device according to (11). (13) A quantization unit that quantizes the coefficient data orthogonally transformed by the orthogonal transformation unit, The encoding unit encodes the coefficient data quantized by the quantization unit. The image processing device according to (11) or (12). (14) A calculation unit for generating a residual between the image and a predicted image, The orthogonal transform unit performs an orthogonal transform on the residual coefficient data. An image processing device according to any one of (10) to (13). (15) Setting a transformation type resulting in a minimum coding cost in the non-joint chrominance coding mode as a transformation type in the joint chrominance coding mode, and deriving a coding cost in the joint chrominance coding mode, thereby setting a coding mode for coding an image. Image processing methods. [Explanation of symbols]

[0206] 300 image encoding device, 301 control unit, 312 calculation unit, 313 orthogonal transformation unit, 314 quantization unit, 315 encoding unit

Claims

1. a setting unit for setting a conversion type in a non-joint chrominance encoding mode as a conversion type in a joint chrominance encoding mode; an orthogonal transform unit that performs orthogonal transform on coefficient data of an image according to a transform type in the joint chrominance encoding mode set by the setting unit; An image processing device comprising:

2. The setting unit sets a transform skip as the transform type in the joint chrominance encoding mode. The image processing device according to claim 1 .

3. The setting unit applies DCT2 as the transform type in the joint chrominance encoding mode when the transform skip is not applied. The image processing device according to claim 2 .

4. The setting unit sets a value of a transform skip flag in the non-joint chrominance encoding mode as a value of the transform skip flag in the joint chrominance encoding mode. The image processing device according to claim 2 .

5. The setting unit sets the joint chrominance encoding mode based on a chrominance encoding block flag in the non-joint chrominance encoding mode. The image processing device according to claim 1 .

6. The setting unit sets an encoding component identifier based on the joint chrominance encoding mode. The image processing device according to claim 5 .

7. The setting unit derives a coding cost of each transform type in the non-joint chrominance coding mode, sets the transform type having the smallest coding cost among the derived transform types, and sets the set transform type as the transform type in the joint chrominance coding mode. The image processing device according to claim 1 .

8. Setting a transform type in a non-joint chrominance encoding mode as a transform type in a joint chrominance encoding mode; orthogonally transforming coefficient data of an image according to a transformation type in the set joint color difference encoding mode; An image processing method comprising:

Images